BASIC OS for the PC

classic Classic list List threaded Threaded
27 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

BASIC OS for the PC

Baltissen, GJPAA (Ruud)

Hallo allemaal,

 

 

"Basic OS for the PC" is a project that started three years ago just as a fun project because I was a bit fed up with the 6502 at that  moment. The idea was to make an OS that uses BASIC as base, just like the C64 does, and it should be able to run on an 8088 machine with only one 360 KB floppy, like the Commodore PC1. It still is an "in between" project and therefore there is only a slow (but steady) progress.

 

So far I developed my own file system (FS), but one based on Commodore's one with BAM and linking bytes in every disk sector. I added my own things like a time stamp and not using two bytes as link but four to overcome the 16 MB limit of Commodore's two-bytes link system. But having your own FS has one disadvantage: it makes things less accessible for others. So I decided to use FAT12 as well . FAT16 is something for the future when hard disk drive are going to play a role.

Note: the BASIC part I developed so far runs with both FSes without changing even one character. A matter of good modular programming I think :)

 

All my old IBM PC/XTs have at least 256 KB of RAM so I decided not to limit myself to 64 KB (or less) as the C64 does but to use the RAM more in the way the C128 or a CBM-II does: 64 KB for the program and 64 KB for the data. The next step in the future is evident: using more than 64 KB for both parts of course. The length of variables won't be limited to just two characters and I think 16 chars is reasonable.

 

Commodore BASIC uses tokens to save the original code in memory and on disk. One of my idea was to skip the tokens. Because I can skip the conversion then, it simplifies saving the program lines and listing them again. The greatest disadvantage however is that I will loose time when running the program. So I will stick to the tokenizing. (unless somebody has a better idea)

 

Another advantage of using tokens is that IMHO I should be able to run Commodore PRGs without any conversion.

 

Commodore stores variables in programs by storing the complete name. Another idea I have is to replace the name by three bytes: one token and two bytes. These two bytes point directly to the data. Advantage: no search for the data needed anymore. Disadvantage: I loose compatibility with Commodore's PRG.

 

 

I also have been playing with my own DOS. The look and feel should be like MSDOS so I won't alienate people right from the start. But other things will be completely different. I assume that you are all familiar with the DOS commands DIR, TIME and DATE. In MSDOS these commands are incorporated in COMMAND.COM. In my case they are separate programs like DEBUG and CHKDSK. It would be nice if I'm able to run original MSDOS programs in the end but I'm realistic enough to know that that will remain a dream most probably. But you never know :)

 

Comments are welcome!

 

Met vriendelijke groet / With kind regards,

Ruud Baltissen

www.Baltissen.org

 

 

 



De informatie in dit e-mailbericht is vertrouwelijk en uitsluitend bestemd voor de
geadresseerde. Wanneer u dit bericht per abuis ontvangt, verzoeken wij u contact op te
nemen met de afzender per kerende e-mail. Verder verzoeken wij u in dat geval dit
e-mailbericht te vernietigen en de inhoud ervan aan niemand openbaar te maken.
Wij aanvaarden geen aansprakelijkheid voor onjuiste, onvolledige dan wel ontijdige
overbrenging van de inhoud van een verzonden e-mailbericht, noch voor daarbij
overgebrachte virussen.

APG Groep N.V. is gevestigd te Heerlen en is ingeschreven in het
handelsregister van de Kamer van Koophandel Limburg onder nummer 14099617


The information contained in this e-mail is confidential and may be privileged.
It may be read, copied and used only by the intended recipient.
If you have received it in error, please contact the sender immediately by
return e-mail; please delete in this case the e-mail and do not disclose its
contents to any person. We don't accept liability for any errors, omissions,
delays of receipt or viruses in the contents of this message which arise as a
result of e-mail transmission.

APG Groep N.V. is registered in the trade register of the Chamber
of Commerce Limburg, The Netherlands, registration number: 14099617
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BASIC OS for the PC

Spiro Trikaliotis
Hello Ruud,

as always, I will not ask for the "why" and just go directly into the
details. ;)

* On Tue, Dec 20, 2016 at 10:35:35AM +0000 Baltissen, GJPAA (Ruud) wrote:
 
> "Basic OS for the PC" is a project that started three years ago just as a fun
> project because I was a bit fed up with the 6502 at that  moment. The idea was
> to make an OS that uses BASIC as base, just like the C64 does, and it should be
> able to run on an 8088 machine with only one 360 KB floppy, like the Commodore
> PC1. It still is an "in between" project and therefore there is only a slow
> (but steady) progress.

Is it written in 8086/8088 assembler, or in a high-level language?

Is the source available somewhere? ;)

> Commodore BASIC uses tokens to save the original code in memory and on disk.
> One of my idea was to skip the tokens. Because I can skip the conversion then,
> it simplifies saving the program lines and listing them again. The greatest
> disadvantage however is that I will loose time when running the program. So I
> will stick to the tokenizing. (unless somebody has a better idea)

I believe two formats, one "in memory format" (with tokens), and one
"on disk format", plain-text when storing the program externally, should
give you more advantage.

You can use any editor externally, but you can also do any optimisations
you like internally. You can even change the in-memory-format as often
as you like, as it will not affect the on disk format.

> Another advantage of using tokens is that IMHO I should be able to run
> Commodore PRGs without any conversion.

You could "automagically" recognise CBM BASIC programs (start address
with a $01 as first byte, and "reasonable" linking addresses before each
line) and convert it "on the fly" to your in-memory-format.

> Commodore stores variables in programs by storing the complete name. Another
> idea I have is to replace the name by three bytes: one token and two bytes.
> These two bytes point directly to the data. Advantage: no search for the data
> needed anymore. Disadvantage: I loose compatibility with Commodore's PRG.

See above, the disadvantage is not as big if you use a specific
in-memory-format that does not need to be identical to the external
format(s).

You could also add a destination address for GOTO/GOSUBs, so you do not
have to search the line targets over and over again).


Regards,
Spiro.

--
Spiro R. Trikaliotis
http://www.trikaliotis.net/

       Message was sent through the cbm-hackers mailing list
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BASIC OS for the PC

Michał Pleban
In reply to this post by Baltissen, GJPAA (Ruud)
Hello!

Baltissen, GJPAA (Ruud) wrote:

> So far I developed my own file system (FS), but one based on Commodore's
> one with BAM and linking bytes in every disk sector. I added my own
> things like a time stamp and not using two bytes as link but four to
> overcome the 16 MB limit of Commodore's two-bytes link system.

Two things which are quite annoying in that kind of filesystem:

* The sectors not hold odd number of bytes - in case of Commodore, 254
instead of 256.

* It is difficult to implement file seek operation - you need to read
every sector in a file up to the desired location to seek (unless you
implement more sophisticated moechanisms like REL files).

For these reasons, I think FAT is much better filesystem.

> All my old IBM PC/XTs have at least 256 KB of RAM so I decided not to
> limit myself to 64 KB (or less) as the C64 does but to use the RAM more
> in the way the C128 or a CBM-II does: 64 KB for the program and 64 KB
> for the data. The next step in the future is evident: using more than 64
> KB for both parts of course. The length of variables won't be limited to
> just two characters and I think 16 chars is reasonable.

You can do the same as the CBM-II 256: first 64 kB bank for the program,
second for string variables, third for numeric variables, and fourth for
arrays.

> Commodore stores variables in programs by storing the complete name.
> Another idea I have is to replace the name by three bytes: one token and
> two bytes. These two bytes point directly to the data.

And what happens when a variable gets deleted? Is there a garbage
collection in the variable area? If yes, then the addresses of the
variables after the deleted one will be changed. You will then need to
change these addresses in the program code as well. Looks complicated.

> I also have been playing with my own DOS. The look and feel should be
> like MSDOS so I won't alienate people right from the start. But other
> things will be completely different. I assume that you are all familiar
> with the DOS commands DIR, TIME and DATE. In MSDOS these commands are
> incorporated in COMMAND.COM. In my case they are separate programs like
> DEBUG and CHKDSK.

Well then, don't forget things like DEL, REN, TYPE and so on - they are
quite useful too ;-)

Regards,
Michau.

       Message was sent through the cbm-hackers mailing list
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BASIC OS for the PC

David Wood-2
linked-list filesystems like FAT are great in that they allow full sectors to be used for data.  They also make it easier to seek within a file- even though FAT doesnt support reverse search, it's relatively trivial to find a specific position within a file no matter the current position.

Another method would be to maintain a BAM as well as a linked list on a per-file basis.  This would be helpful in that sparse files could be possible, which would be great when dealing with random access;  the middle of a file won't have to be created to place a record near the end, and the whole file wouldn't even need to be allocated.

Have a look at other 8-bit systems and how they handled their filesystems- apples had an interesting way of doing filesystems without sector linking in their original DOS release, using the link list method I mentioned.  Their later DOS expanded on that by increasing the linked lists to a tree format (a linked list of linked lists), that works remarkably well even at 1MHz.  It's a big shame what they did with filename limtations though.  That however is a directory format issue; I really like their metadata format. :)


On Tue, Dec 20, 2016 at 8:37 AM, Michał Pleban <[hidden email]> wrote:
Hello!

Baltissen, GJPAA (Ruud) wrote:

> So far I developed my own file system (FS), but one based on Commodore's
> one with BAM and linking bytes in every disk sector. I added my own
> things like a time stamp and not using two bytes as link but four to
> overcome the 16 MB limit of Commodore's two-bytes link system.

Two things which are quite annoying in that kind of filesystem:

* The sectors not hold odd number of bytes - in case of Commodore, 254
instead of 256.

* It is difficult to implement file seek operation - you need to read
every sector in a file up to the desired location to seek (unless you
implement more sophisticated moechanisms like REL files).

For these reasons, I think FAT is much better filesystem.

> All my old IBM PC/XTs have at least 256 KB of RAM so I decided not to
> limit myself to 64 KB (or less) as the C64 does but to use the RAM more
> in the way the C128 or a CBM-II does: 64 KB for the program and 64 KB
> for the data. The next step in the future is evident: using more than 64
> KB for both parts of course. The length of variables won't be limited to
> just two characters and I think 16 chars is reasonable.

You can do the same as the CBM-II 256: first 64 kB bank for the program,
second for string variables, third for numeric variables, and fourth for
arrays.

> Commodore stores variables in programs by storing the complete name.
> Another idea I have is to replace the name by three bytes: one token and
> two bytes. These two bytes point directly to the data.

And what happens when a variable gets deleted? Is there a garbage
collection in the variable area? If yes, then the addresses of the
variables after the deleted one will be changed. You will then need to
change these addresses in the program code as well. Looks complicated.

> I also have been playing with my own DOS. The look and feel should be
> like MSDOS so I won't alienate people right from the start. But other
> things will be completely different. I assume that you are all familiar
> with the DOS commands DIR, TIME and DATE. In MSDOS these commands are
> incorporated in COMMAND.COM. In my case they are separate programs like
> DEBUG and CHKDSK.

Well then, don't forget things like DEL, REN, TYPE and so on - they are
quite useful too ;-)

Regards,
Michau.

       Message was sent through the cbm-hackers mailing list

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BASIC OS for the PC

Marko Mäkelä
In reply to this post by Baltissen, GJPAA (Ruud)
Hi Ruud,

>"Basic OS for the PC" is a project that started three years ago just as
>a fun project because I was a bit fed up with the 6502 at that  moment.
>The idea was to make an OS that uses BASIC as base, just like the C64
>does, and it should be able to run on an 8088 machine with only one 360
>KB floppy, like the Commodore PC1. It still is an "in between" project
>and therefore there is only a slow (but steady) progress.

This somehow reminds me on TempleOS, which is a 32-bit or 64-bit
operating system, all written by a single programmer.  
http://www.templeos.org/Wb/Doc/FAQ.html

(This whole hobby also reminds me of
http://www.searchquotes.com/search/Genius_And_Insanity,/ or something
similar.)

Now that virtual machines and also x86 emulators are common, it could be
interesting to write own operating systems for x86. But I guess on this
group we would want to concentrate on the simpler CPUs with no MMU or
virtual memory, that is, at most 8086 or 80286. :)

>Commodore BASIC uses tokens to save the original code in memory and on
>disk. One of my idea was to skip the tokens. Because I can skip the
>conversion then, it simplifies saving the program lines and listing
>them again. The greatest disadvantage however is that I will loose time
>when running the program. So I will stick to the tokenizing. (unless
>somebody has a better idea)
>
>Another advantage of using tokens is that IMHO I should be able to run
>Commodore PRGs without any conversion.

Like Spiro, I would propose using a separate main-memory representation
("byte-code"). There could be multiple serializers and deserializers for
different on-disk formats (plain text, Commodore tokenized), chosen by a
parameter to LOAD or SAVE. I am not sure if the interpreter really needs
to incorporate an editor. (So, maybe there should not be a SAVE command
either, except maybe when used for copying LOADed files.)

Speaking of cross-platform programming, did anyone else enjoy the CppCon
2016 talks on YouTube? There were many interesting lectures. I posted
https://github.com/lefticus/x86-to-6502/issues/2 to reduce the C64
program file size of the toy Pong game presented in
https://www.youtube.com/watch?v=zBkNBP00wJE

On related news, I would like to help with
https://github.com/beholdnec/llvm-m6502 to create a modern C/C++
compiler for the 6502, if only I had more spare time. I do not think
that this is completely unreasonable, because there already is avr-gcc
(targeting an 8-bit microcontroller family) and also
https://github.com/avr-llvm/llvm/ in the works. Clang seems to be a
little better at C++ and high-level code transformations and
whole-program optimizations than GCC, which could benefit small
platforms.

        Marko

       Message was sent through the cbm-hackers mailing list
smf
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BASIC OS for the PC

smf
In reply to this post by David Wood-2
On 20/12/2016 15:05, David Wood wrote:
> Another method would be to maintain a BAM as well as a linked list on a
> per-file basis.  This would be helpful in that sparse files could be
> possible, which would be great when dealing with random access;  the middle
> of a file won't have to be created to place a record near the end, and the
> whole file wouldn't even need to be allocated.

I think you could do sparse files with FAT easier by just using a magic
entry in the list for "no space allocated yet".

I think it's better to keep it compatible though, because as soon as you
try to improve things there will be 100 things you'd want to do better
which would probably blow your code size budget & you won't be
compatible with anything.

FAT and BASIC also have history...

https://en.wikipedia.org/wiki/File_Allocation_Table#Original_8-bit_FAT

I had thought about porting http://www.pagetable.com/?p=48 to UEFI,
which would serve a similar(ish) purpose to this.


       Message was sent through the cbm-hackers mailing list
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: BASIC OS for the PC

Baltissen, GJPAA (Ruud)
In reply to this post by Spiro Trikaliotis
Hallo Spiro,


> as always, I will not ask for the "why" and just go directly into the details. ;)

In fact it was triggered by a Linux guru here at work who said that creating an OS was doable for one person. And I must say it is best a bit of fun. I certainly learned a lot about The BIOS, FAT and other disk related material.


> Is it written in 8086/8088 assembler, or in a high-level language?

Assembly.


> Is the source available somewhere? ;)

Not on Internet, yet. But you are free to send an email to get the stuff. I must warn you: although it is well documented (IMHO), it is still quite a lot of rough material with lots of try (and error) code.


> I believe two formats, one "in memory format" (with tokens), and one "on disk format",
> plain-text when storing the program externally, should give you more advantage.
> You can use any editor externally, but you can also do any optimizations you like internally.
> You can even change the in-memory-format as often as you like, as it will not affect the on
> disk format.

Great idea! It does mean we need a conversion the moment the program is loaded but I think the mentioned advantages out weight the loss of time.


> You could "automagically" recognise CBM BASIC programs (start address...

Hmm, I completely forgot about the start address and I even don't use it. So I wouldn't be compatible anyway.


> You could also add a destination address for GOTO/GOSUBs, so you do not have to search the line targets over and over again.

Another good idea!



Hello Michał,


> For these reasons, I think FAT is much better filesystem.

I agree. But the main reason for using this CBM-FS was that I wanted something that wasn't MSDOS related. Another reason: because of my work with the various C= drives I was more than familiar with it. That I switched to FAT now doesn't mean that I abandoned it. It is just a matter of two different files that handle all file access and a directive in the source that tells the compiler which one of the two to use.
But its best purpose: it showed some people at my work to think a bit out of their well known box.

Edit: I wrote the above yesterday. Your remark about the 254 bytes kept ringing a bell. When I checked things in the evening I found out why. I still use the link bytes but not in the sectors them self but in the BAM. In fact something like the FAT does. There was a very simple reason to do this. Now I can load a sector directly into memory. Otherwise I had to load the sector in memory first, strip the link bytes and copy the rest to the end of the already loaded part. That meant handling the data twice. And it would also mean I couldn't load more sectors in one go.
For a drive these link bytes are no problem because the only thing that it will do with the loaded data is to send it to the connected computer. The moment that is done, the sector in memory is overwritten by a new one.
One extra detail, I use two sets of link bytes: one set points forward, one points backwards. It enables one to search back in the chain, if needed. For floppies I use two bytes for each link, for hard disks four. That is enough for 128 GB.


> You can do the same as the CBM-II 256: first 64 kB bank for the program, second for string variables, third for numeric variables, and fourth for arrays.

I didn't know about this detail and I'll keep it in my mind.


> Is there a garbage collection in the variable area?

Very good question. So far none. But there is a simple reason for that: I still don't understand what garbage collection exactly does. In my understanding, if I understood Wikipedia correctly, the most less used variables are disposed. But in my opinion that is crazy. And AFAIK Pascal and C don't have garbage collection either. Why not if it is such a "good" tool?
If someone is willing to explain in other words what garbage collection exactly does, what its benefits plus the assurance that it won't jeopardize any program, I might install it in time (in 10 years or so :)


> Well then, don't forget things like DEL, REN, TYPE and so on - they are quite useful too ;-)

I know, so I won't :)


Marko:> guess on this group we would want to concentrate on the simpler CPUs with no MMU or virtual memory, that is, at most 8086 or 80286. :)

That's exactly what I had in mind. For 386+ you have Linux, Minix and other OSes. The first comment I got when explaining that I was targeting the 8088: "but then you won't have multi-tasking, multi-user and no memory protection!". My God, at that time I didn't even know how a boot loader worked.
For the moment I'm happy with my simple 8088 :)


FYI: tomorrow we will go to Poland to spend Christmas there. And my parents-in-law don't have internet. So don't expect any replies from me for the time being. Which leaves me one thing to say:

         Merry Christmas and Happy New Year !!!


Met vriendelijke groet,
Ruud Baltissen












De informatie in dit e-mailbericht is vertrouwelijk en uitsluitend bestemd voor de
geadresseerde. Wanneer u dit bericht per abuis ontvangt, verzoeken wij u contact op te
nemen met de afzender per kerende e-mail. Verder verzoeken wij u in dat geval dit
e-mailbericht te vernietigen en de inhoud ervan aan niemand openbaar te maken.
Wij aanvaarden geen aansprakelijkheid voor onjuiste, onvolledige dan wel ontijdige
overbrenging van de inhoud van een verzonden e-mailbericht, noch voor daarbij
overgebrachte virussen.

APG Groep N.V. is gevestigd te Heerlen en is ingeschreven in het
handelsregister van de Kamer van Koophandel Limburg onder nummer 14099617


The information contained in this e-mail is confidential and may be privileged.
It may be read, copied and used only by the intended recipient.
If you have received it in error, please contact the sender immediately by
return e-mail; please delete in this case the e-mail and do not disclose its
contents to any person. We don't accept liability for any errors, omissions,
delays of receipt or viruses in the contents of this message which arise as a
result of e-mail transmission.

APG Groep N.V. is registered in the trade register of the Chamber
of Commerce Limburg, The Netherlands, registration number: 14099617


       Message was sent through the cbm-hackers mailing list
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: BASIC OS for the PC

Baltissen, GJPAA (Ruud)
Rereading doesn't always work out well:


> In fact it was triggered by a Linux guru here at work who said that creating an OS was doable for one person.

Should have been:
In fact it was triggered by a Linux guru here at work who said that creating an OS was NOT doable for one person.


Ruud
De informatie in dit e-mailbericht is vertrouwelijk en uitsluitend bestemd voor de
geadresseerde. Wanneer u dit bericht per abuis ontvangt, verzoeken wij u contact op te
nemen met de afzender per kerende e-mail. Verder verzoeken wij u in dat geval dit
e-mailbericht te vernietigen en de inhoud ervan aan niemand openbaar te maken.
Wij aanvaarden geen aansprakelijkheid voor onjuiste, onvolledige dan wel ontijdige
overbrenging van de inhoud van een verzonden e-mailbericht, noch voor daarbij
overgebrachte virussen.

APG Groep N.V. is gevestigd te Heerlen en is ingeschreven in het
handelsregister van de Kamer van Koophandel Limburg onder nummer 14099617


The information contained in this e-mail is confidential and may be privileged.
It may be read, copied and used only by the intended recipient.
If you have received it in error, please contact the sender immediately by
return e-mail; please delete in this case the e-mail and do not disclose its
contents to any person. We don't accept liability for any errors, omissions,
delays of receipt or viruses in the contents of this message which arise as a
result of e-mail transmission.

APG Groep N.V. is registered in the trade register of the Chamber
of Commerce Limburg, The Netherlands, registration number: 14099617


       Message was sent through the cbm-hackers mailing list
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BASIC OS for the PC

Marko Mäkelä
Hi Ruud,

On Wed, Dec 21, 2016 at 07:08:30AM +0000, Baltissen, GJPAA (Ruud) wrote:
>In fact it was triggered by a Linux guru here at work who said that
>creating an OS was NOT doable for one person.

Linus Torvalds is another counterexample to that.

I recently read his book "Just for fun" (or actually the Finnish
translation by his sister). It certainly was a huge task to implement
enough system calls to get /bin/sh running. Other major early
accomplishments include the VT-100 emulator that still exists in the
Linux text console and the VFS layer and the first file systems. I guess
that the next major accomplishment after /bin/sh was to get GCC running
on Linux, so that he no longer had to boot to Minix for development.

As far as I understand, other contributors entered the picture only
after Linux had become independent of Minix.

(I am not suggesting that your goal should be a self-hosting
environment. I think that cross-compiling makes perfect sense. I guess
nobody develops Android applications on Android, for example.)

Seasons greetings,

        Marko

       Message was sent through the cbm-hackers mailing list
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BASIC OS for the PC

Spiro Trikaliotis
In reply to this post by Baltissen, GJPAA (Ruud)
Hello Ruud,

* On Wed, Dec 21, 2016 at 06:58:50AM +0000 Baltissen, GJPAA (Ruud) wrote:
 
> > as always, I will not ask for the "why" and just go directly into
> > the details. ;)
>
> In fact it was triggered by a Linux guru here at work who said that
> creating an OS was doable for one person. And I must say it is best a
> bit of fun. I certainly learned a lot about The BIOS, FAT and other
> disk related material.

It depends upon what you mean by an OS. It is perfectly doable,
especially if it is somehow special purpose and you do not need to make
it to do everything.
 
> > Is the source available somewhere? ;)
>
> Not on Internet, yet. But you are free to send an email to get the
> stuff. I must warn you: although it is well documented (IMHO), it is
> still quite a lot of rough material with lots of try (and error) code.

You will get an e-mail. ;)

 

> > Is there a garbage collection in the variable area?
>
> Very good question. So far none. But there is a simple reason for
> that: I still don't understand what garbage collection exactly does.
> In my understanding, if I understood Wikipedia correctly, the most
> less used variables are disposed. But in my opinion that is crazy. And
> AFAIK Pascal and C don't have garbage collection either. Why not if it
> is such a "good" tool?  If someone is willing to explain in other
> words what garbage collection exactly does, what its benefits plus the
> assurance that it won't jeopardize any program, I might install it in
> time (in 10 years or so :)

Well, the C64 BASIC has a garbage collection (GC), too.

Think about the string variables: As soon as these are not only
assignments of the form A$="ABC" from the program text, in which case
the pointer points into the program area, the strings are generated on
the string area (the end of the basic memory, that is, from $A000 going
downwards). Thus, the string variable pointer points to the string
there.

Now, you do string manipulations (for example, something like A$ =
B$+B$, or then B$=something else). After these transformatios, the old
values of A$ (first assignment) or B$ (second assignment) or not needed
anymore. In CBM BASIC, these "old" strings remain as garbase in the
string area.

If you do many of these manipulations, then the end of the string area
(which grows down, remember) will eventually hit the end of the variable
(array) area. When this happens, the BASIC could output "?OUT OF MEMORY
ERROR"

But wait: There are possibly many strings in the string area that are
only garbage. Thus, the BASIC starts a GC and tries to stuff all the
strings together so that the garbage does not occupy any memory anymore.
For this, it looks which strings are still used (by checking which
pointers to the string area still exist), and removing everything in
between.

Unfortunately, the implementation in the CBM BASIC (MS BASIC) has a
complexity of O(n^2), that's the reason why this can take much time (I
have had cases in which it needed half an hour! In this time, you
believe that the computer has crashed, as there is no progress at all).

Only if after the GC, there is still not enough room, you get the "?OUT
OF MEMORY ERROR", and the program stops.

The GC in the C64 BASIC starts at $B526, btw.

Starting with the 264 series, CBM used a better approach for GC. In
order to allow for this, every string in the string area contained a
"back link" to the string descriptor, if it was still used, or a special
sign (0 byte, IIRC) and the length of the garbage area, so the algorithm
could run in O(n) instead.


Note that the GC in BASIC does NOT try to remove the 7 byte variable
memory that is generated for every variable after the BASIC area (or the
x byte for any array). The benefit is not much, and you could only
remove variables that are already "restored" to the base value (0 for
numeric ones, empty string for strings). One could do this, but I am not
sure if the benefit is high enough to justify this.
 
> Marko:> guess on this group we would want to concentrate on the
> simpler CPUs with no MMU or virtual memory, that is, at most 8086 or
> 80286. :)
>
> That's exactly what I had in mind. For 386+ you have Linux, Minix and
> other OSes.

My first attempt on an OS was exactly the opposite, I wanted to create a
386 based OS. I did not go very far, though, because of lack of time. :(

> FYI: tomorrow we will go to Poland to spend Christmas there. And my
> parents-in-law don't have internet. So don't expect any replies from
> me for the time being. Which leaves me one thing to say:

No internet? How that? ;)

>          Merry Christmas and Happy New Year !!!

Enjoy your free time!

Regards,
Spiro.

--
Spiro R. Trikaliotis
http://www.trikaliotis.net/

       Message was sent through the cbm-hackers mailing list
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BASIC OS for the PC

Michał Pleban
In reply to this post by Baltissen, GJPAA (Ruud)
Hello!

Baltissen, GJPAA (Ruud) wrote:

> Edit: I wrote the above yesterday. Your remark about the 254 bytes kept ringing a bell. When I checked things in the evening I found out why. I still use the link bytes but not in the sectors them self but in the BAM. In fact something like the FAT does.

So yes, basically you don't have a BAM anymore but more something like
FAT. So your filesystem is not that similar to CBM after al ;-)

> Very good question. So far none. But there is a simple reason for that: I still don't understand what garbage collection exactly does. I

Maybe garbage collection is not the best term. I think "memory
defragmentation" might be better.

Basically, when you create string variables in memory, they take some
space. For example:

A$ = "aaaaaaaaaaaaaaaa"
B$ = "bbbbbbbbbbbbbbbb"

Suppose you later change one variable:

A$ = "cccccccc"
B$ = "bbbbbbbbbbbbbbbb"

Now you have 8 free bytes between A$ and B$. If you want to create a
third variable which is 8 bytes long, it's fine. But a 9-byte variable
will not fit there so the free space remains unused.

As the program is running and changing its variables, there are more and
more such small free holes in memory. You need a program that compacts
all these small holes into one large free space.

> For the moment I'm happy with my simple 8088 :)

Will it run on a CBM-II with a 8088 card? :-P

Regards,
Michau.

       Message was sent through the cbm-hackers mailing list
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: BASIC OS for the PC

Baltissen, GJPAA (Ruud)
In reply to this post by Spiro Trikaliotis
Hallo Sipro,


> Think about the string variables: As soon as these are not only assignments ....

Very very good explanation, thank you very much!

My idea was quite simple: the moment I run into a variable, I will reserve memory for it in the data area. And just like in Pascal, 256 bytes for the string. That is, purely for the data. I also have to reserve some memory for the name, type, etc. That is not needed for a compiled program but an interpreter has to check every time if a found variable already exists or not.


> in which case the pointer points into the program area

Hmm, nice trick. Can save a lot of memory. Just need to come up with something smart for what to do if the string is changed. I think a simple pointer will do: it either points to the program area or the reserved area. But then I need a mechanism that tells BASIS what memory is in use for strings or not. Just a quick one: non-strings bottom-up, strings top-down plus a pointer that points to the free area under the string-memory. But the moment they meet each other I have an "?OUT OF MEMORY ERROR".


> No internet? How that? ;)

Quite simple, they don't want it/haven't any use for it. FYI, they live in Górki Noteckie (use Google Maps to find it) and so far I haven't found any Wifi signal there yet. I cannot imagine that there isn't any internet at all (satellite, internet over mobile phone) there but still a bit strange.


Met vriendelijke groet / With kind regards,
Ruud Baltissen

www.Baltissen.org



De informatie in dit e-mailbericht is vertrouwelijk en uitsluitend bestemd voor de
geadresseerde. Wanneer u dit bericht per abuis ontvangt, verzoeken wij u contact op te
nemen met de afzender per kerende e-mail. Verder verzoeken wij u in dat geval dit
e-mailbericht te vernietigen en de inhoud ervan aan niemand openbaar te maken.
Wij aanvaarden geen aansprakelijkheid voor onjuiste, onvolledige dan wel ontijdige
overbrenging van de inhoud van een verzonden e-mailbericht, noch voor daarbij
overgebrachte virussen.

APG Groep N.V. is gevestigd te Heerlen en is ingeschreven in het
handelsregister van de Kamer van Koophandel Limburg onder nummer 14099617


The information contained in this e-mail is confidential and may be privileged.
It may be read, copied and used only by the intended recipient.
If you have received it in error, please contact the sender immediately by
return e-mail; please delete in this case the e-mail and do not disclose its
contents to any person. We don't accept liability for any errors, omissions,
delays of receipt or viruses in the contents of this message which arise as a
result of e-mail transmission.

APG Groep N.V. is registered in the trade register of the Chamber
of Commerce Limburg, The Netherlands, registration number: 14099617


       Message was sent through the cbm-hackers mailing list
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: BASIC OS for the PC

Baltissen, GJPAA (Ruud)
In reply to this post by Michał Pleban
Hallo Michał,


> Maybe garbage collection is not the best term. I think "memory defragmentation" might be better.

After reading Spiro's and your explanation, indeed a much better term.


> Will it run on a CBM-II with a 8088 card? :-P

Never thought about that, a very nice idea! But that will involve some extra work because of the 256 bytes sectors. It becomes time to dig up one of my 710s.


Met vriendelijke groet / With kind regards,
Ruud Baltissen

www.Baltissen.org




De informatie in dit e-mailbericht is vertrouwelijk en uitsluitend bestemd voor de
geadresseerde. Wanneer u dit bericht per abuis ontvangt, verzoeken wij u contact op te
nemen met de afzender per kerende e-mail. Verder verzoeken wij u in dat geval dit
e-mailbericht te vernietigen en de inhoud ervan aan niemand openbaar te maken.
Wij aanvaarden geen aansprakelijkheid voor onjuiste, onvolledige dan wel ontijdige
overbrenging van de inhoud van een verzonden e-mailbericht, noch voor daarbij
overgebrachte virussen.

APG Groep N.V. is gevestigd te Heerlen en is ingeschreven in het
handelsregister van de Kamer van Koophandel Limburg onder nummer 14099617


The information contained in this e-mail is confidential and may be privileged.
It may be read, copied and used only by the intended recipient.
If you have received it in error, please contact the sender immediately by
return e-mail; please delete in this case the e-mail and do not disclose its
contents to any person. We don't accept liability for any errors, omissions,
delays of receipt or viruses in the contents of this message which arise as a
result of e-mail transmission.

APG Groep N.V. is registered in the trade register of the Chamber
of Commerce Limburg, The Netherlands, registration number: 14099617
1�,j�j���a�����^q��i��ɚ�X��X��
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BASIC OS for the PC

silverdr-2
In reply to this post by Baltissen, GJPAA (Ruud)

> On 2016-12-21, at 07:58, Baltissen, GJPAA (Ruud) <[hidden email]> wrote:
>
>> I believe two formats, one "in memory format" (with tokens), and one "on disk format",
>> plain-text when storing the program externally, should give you more advantage.
>> You can use any editor externally, but you can also do any optimizations you like internally.
>> You can even change the in-memory-format as often as you like, as it will not affect the on
>> disk format.
>
> Great idea! It does mean we need a conversion the moment the program is loaded but I think the mentioned advantages out weight the loss of time.

Definitely, IMHO.

>> You could "automagically" recognise CBM BASIC programs (start address...
>
> Hmm, I completely forgot about the start address and I even don't use it. So I wouldn't be compatible anyway.

Well, no... ? "if by "start address" you mean the "load address", it is not a feature of the CBM BASIC but rather CBM DOS, isn't it? Meaning it doesn't affect the run-time compatibility.

>> Is there a garbage collection in the variable area?
>
> Very good question. So far none. But there is a simple reason for that: I still don't understand what garbage collection exactly does.

It's more complex than that but to keep it simple: GC frees the memory allocated by stuff your program no longer needs so that stuff your program is actually still going to need can get the memory when needed.

> In my understanding, if I understood Wikipedia correctly, the most less used variables are disposed. But in my opinion that is crazy.

Not at all ;-) Epsecially when we talk about system with highly limited [memory] resources.

> And AFAIK Pascal and C don't have garbage collection either. Why not if it is such a "good" tool?

C is a compiled, static typed, "close to metal" type of language. Lots of things, including memory requirements are resolved at compile time. The remaining you have to take care of yourself (both allocating and freeing). Even with that you can still use GC with C. It's just that the evolution of the standard of the language never went that way. I am not the right one to ask about Pascal as I possibly can't say a single good word about it ;-)

> If someone is willing to explain in other words what garbage collection exactly does, what its benefits plus the assurance that it won't jeopardize any program, I might install it in time (in 10 years or so :)

In small systems and non-OO languages it is much simpler than what we usually refer to when saying GC. Michau gave a good example.

> FYI: tomorrow we will go to Poland to spend Christmas there. And my parents-in-law don't have internet. So don't expect any replies from me for the time being. Which leaves me one thing to say:
>
>         Merry Christmas and Happy New Year !!!


The same to you, and… consider yourself invited as always!

--
SD!


       Message was sent through the cbm-hackers mailing list
smf
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BASIC OS for the PC

smf
In reply to this post by Baltissen, GJPAA (Ruud)
On 21/12/2016 11:09, Baltissen, GJPAA (Ruud) wrote
>> Maybe garbage collection is not the best term. I think "memory defragmentation" might be better.
> After reading Spiro's and your explanation, indeed a much better term.

Garbage Collection is the correct term though.

https://en.wikipedia.org/wiki/Garbage_collection_(computer_science)#BASIC

It's very similar to the C# and Java concept of Garbage collection as well.


       Message was sent through the cbm-hackers mailing list
smf
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BASIC OS for the PC

smf
In reply to this post by silverdr-2
On 21/12/2016 12:49, [hidden email] wrote:
>> Great idea! It does mean we need a conversion the moment the program is loaded but I think the mentioned advantages out weight the loss of time.
> Definitely, IMHO.

I was using BBCBASIC(86) on PC's way past it's sell by date and I ended
up storing all the source files as text in subversion and then
converting them into tokenised versions to actually run them. As part of
this I also added the ability to include another file, so that you could
have a library of routines shared between modules (we used a version
that was limited to 64k of program and 64k of data). I don't know if
it's in use today, but I think it was still running when Vista came out.

We were kinda stuck with the way it worked, as we didn't have control
over BBCBASIC(86). But as you do then I'd recommend loading and editing
the basic program as text, then when you type RUN then push the code
through something like lvvm to give you native code.

Also put in place something that can load and save native code modules,
so that you don't have to even have the source file. Oh and replace line
numbers with labels.


       Message was sent through the cbm-hackers mailing list
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BASIC OS for the PC

Ruud
Hallo smf,


> Oh and replace line numbers with labels.

Nope, I will keep the line numbers just to retain the original
feeling of BASIC.


--
   
Kind regards / Met vriendelijke groet, Ruud Baltissen
www.Baltissen.org







       Message was sent through the cbm-hackers mailing list
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BASIC OS for the PC

Michał Pleban
In reply to this post by Baltissen, GJPAA (Ruud)
Hello!

Baltissen, GJPAA (Ruud) wrote:

> Quite simple, they don't want it/haven't any use for it. FYI, they live in Górki Noteckie (use Google Maps to find it) and so far I haven't found any Wifi signal there yet. I cannot imagine that there isn't any internet at all (satellite, internet over mobile phone) there but still a bit strange.

Mobile phone + mobile hotspot application should do the trick :-)

Regards,
Michau.


       Message was sent through the cbm-hackers mailing list
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BASIC OS for the PC

Michał Pleban
In reply to this post by smf
Hello!

smf wrote:

> Garbage Collection is the correct term though.
> https://en.wikipedia.org/wiki/Garbage_collection_(computer_science)#BASIC
> It's very similar to the C# and Java concept of Garbage collection as well.

I am still not sure that garbage collection is the corect term here. At
the top of the Wikipedia article, it says:

"The garbage collector attempts to reclaim memory occupied by objects
that are no longer in use by the program."

That's not really what the BASIC does. In Java, Perl etc. you have
reference counts for objects. If the object is no longer referenced
(i.e. there is no code left that uses the object), garbage collector
removes the object and frees the allocated memory. This is in contrast
to languages like C++ where you are responsible to remove all unused
object yourself.

Nothing like this happend in BASIC. When you define a variable, the
variable lives forever (at least in the simple versions like Commodore
BASIC - I am not speaking about Visual Basic etc). Therefore, the memory
allocated for the variable is never reclaimed automatically when the
variable is not used, for the simple reason that the variable never
becomes "unused" (unless you clear all variabled with CLR command). SO
there is really no garbage collection.

What happens, however, is that the variable can change its length so it
may leave some space after it (if it shrinks) or be moved away to a
different location in memory (if it grows) again leaving a chunk of free
space. This is a natural process and again has nothing to do with
garbage collection. The process then needs to be run which compacts the
variables so that these small unused chunks are joined together to form
one large chunk. It involves moving the variables to new memory
locations but it does not free any "unused" space - the space is free
already, it is just moved around in memory.

Regards,
Michau.



       Message was sent through the cbm-hackers mailing list
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BASIC OS for the PC

Marko Mäkelä
On Wed, Dec 21, 2016 at 07:41:32PM +0100, Michał Pleban wrote:
>What happens, however, is that the variable can change its length so it
>may leave some space after it (if it shrinks) or be moved away to a
>different location in memory (if it grows) again leaving a chunk of
>free space. This is a natural process and again has nothing to do with
>garbage collection. The process then needs to be run which compacts the
>variables so that these small unused chunks are joined together to form
>one large chunk. It involves moving the variables to new memory
>locations but it does not free any "unused" space - the space is free
>already, it is just moved around in memory.

I would say that the correct term for this is defragmentation, or
removing fragmentation.

https://en.wikipedia.org/wiki/Fragmentation_(computing)

Whether we should call it internal or external fragmentation is unclear
to me, because the allocator is so tightly coupled with the application
(the rest of the BASIC interpreter).

But, I remember it being called "garbage collection" back in the day. I
learned about (true) garbage collection in other languages much later.

        Marko

       Message was sent through the cbm-hackers mailing list
12
Loading...