Adding flow control to CC65 up2400 driver.

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Adding flow control to CC65 up2400 driver.

Thom Cherryhomes
Am barrelling through PLATOTerm, and I've found that checking the receive buffer in the main-line code and asserting flow control there doesn't seem to be 100% effective in dealing with what still looks like some buffer overrun.

Given this code here:

Would adding a buffer overflow check and asserting bit 1 of $DD01 to assert hardware flow control work? I'm asking because I know this code is _very_ timing sensitive, and I am not very familiar with the user port serial machinery.

Reply | Threaded
Open this post in threaded view
|

Re: Adding flow control to CC65 up2400 driver.

Ed Spittles
To be clear on the directions, then, it's the 8-bit client which is receiving data, has got too much, or nearly too much, and would like to tell the upstream host to stop sending?

I've seen that same situation using the BBC Micro's serial port and connecting to a PC. The Acorn OS has a buffer threshold parameter, and I found I needed to adjust it. That is, the BBC needs to maintain a lot of space in the buffer, and as soon as it has less then 50 bytes it tells the PC to stop sending. The reason is that the PC can take its time to respond to the request to go quiet, and many more bytes may arrive.

Others have seen the same, and set the free space to 100 bytes:
although I see they still report problems.

It's crucial to have the right cable connections, and a suitably cooperative serial port behaviour on the PC.  It might even depend on the responsiveness of the PC OS.

Ed


On 29 July 2018 at 16:45, Thom Cherryhomes <[hidden email]> wrote:
Am barrelling through PLATOTerm, and I've found that checking the receive buffer in the main-line code and asserting flow control there doesn't seem to be 100% effective in dealing with what still looks like some buffer overrun.

Given this code here:

Would adding a buffer overflow check and asserting bit 1 of $DD01 to assert hardware flow control work? I'm asking because I know this code is _very_ timing sensitive, and I am not very familiar with the user port serial machinery.


Reply | Threaded
Open this post in threaded view
|

Re: Adding flow control to CC65 up2400 driver.

Thom Cherryhomes
Yes, I am currently doing this in the main-line code, checking the size of the ring buffer, and if it passes a given threshold, it asserts bit 1 of $DD01, continues to process the now draining buffer, and toggles the line when it's ready for more data...

This has worked, up until yesterday, when it's now clear that I am spending too much time plotting text (that's a whole other matter), and I am trying to get back that reliability while I try to either find help to get the text routines fixed, OR ultimately work them out myself...but now I am wondering if moving the buffer check and the handshaking into the NMI handler for the bit banging serial driver will give me the temporary stability I need to make this work?

-Thom

On Sun, Jul 29, 2018 at 11:00 AM Ed Spittles <[hidden email]> wrote:
To be clear on the directions, then, it's the 8-bit client which is receiving data, has got too much, or nearly too much, and would like to tell the upstream host to stop sending?

I've seen that same situation using the BBC Micro's serial port and connecting to a PC. The Acorn OS has a buffer threshold parameter, and I found I needed to adjust it. That is, the BBC needs to maintain a lot of space in the buffer, and as soon as it has less then 50 bytes it tells the PC to stop sending. The reason is that the PC can take its time to respond to the request to go quiet, and many more bytes may arrive.

Others have seen the same, and set the free space to 100 bytes:
although I see they still report problems.

It's crucial to have the right cable connections, and a suitably cooperative serial port behaviour on the PC.  It might even depend on the responsiveness of the PC OS.

Ed


On 29 July 2018 at 16:45, Thom Cherryhomes <[hidden email]> wrote:
Am barrelling through PLATOTerm, and I've found that checking the receive buffer in the main-line code and asserting flow control there doesn't seem to be 100% effective in dealing with what still looks like some buffer overrun.

Given this code here:

Would adding a buffer overflow check and asserting bit 1 of $DD01 to assert hardware flow control work? I'm asking because I know this code is _very_ timing sensitive, and I am not very familiar with the user port serial machinery.


Reply | Threaded
Open this post in threaded view
|

Re: Adding flow control to CC65 up2400 driver.

Ed Spittles
Oops, I'm in too deep to help there - hopefully I've helped to clarify the situation. My guess would be yes, the NMI service routine is a good place to act early in checking buffer space and turning off the sender, but yes, it's very critical code and you'd need to find the right place to do that work, and that might be difficult.

Ed


On 29 July 2018 at 17:06, Thom Cherryhomes <[hidden email]> wrote:
Yes, I am currently doing this in the main-line code, checking the size of the ring buffer, and if it passes a given threshold, it asserts bit 1 of $DD01, continues to process the now draining buffer, and toggles the line when it's ready for more data...

This has worked, up until yesterday, when it's now clear that I am spending too much time plotting text (that's a whole other matter), and I am trying to get back that reliability while I try to either find help to get the text routines fixed, OR ultimately work them out myself...but now I am wondering if moving the buffer check and the handshaking into the NMI handler for the bit banging serial driver will give me the temporary stability I need to make this work?

-Thom

On Sun, Jul 29, 2018 at 11:00 AM Ed Spittles <[hidden email]> wrote:
To be clear on the directions, then, it's the 8-bit client which is receiving data, has got too much, or nearly too much, and would like to tell the upstream host to stop sending?

I've seen that same situation using the BBC Micro's serial port and connecting to a PC. The Acorn OS has a buffer threshold parameter, and I found I needed to adjust it. That is, the BBC needs to maintain a lot of space in the buffer, and as soon as it has less then 50 bytes it tells the PC to stop sending. The reason is that the PC can take its time to respond to the request to go quiet, and many more bytes may arrive.

Others have seen the same, and set the free space to 100 bytes:
although I see they still report problems.

It's crucial to have the right cable connections, and a suitably cooperative serial port behaviour on the PC.  It might even depend on the responsiveness of the PC OS.

Ed


On 29 July 2018 at 16:45, Thom Cherryhomes <[hidden email]> wrote:
Am barrelling through PLATOTerm, and I've found that checking the receive buffer in the main-line code and asserting flow control there doesn't seem to be 100% effective in dealing with what still looks like some buffer overrun.

Given this code here:

Would adding a buffer overflow check and asserting bit 1 of $DD01 to assert hardware flow control work? I'm asking because I know this code is _very_ timing sensitive, and I am not very familiar with the user port serial machinery.



Reply | Threaded
Open this post in threaded view
|

Re: Adding flow control to CC65 up2400 driver.

André Fachat
On Sonntag, 29. Juli 2018 17:42:56 CEST Ed Spittles wrote:
> Oops, I'm in too deep to help there - hopefully I've helped to clarify the
> situation. My guess would be yes, the NMI service routine is a good place
> to act early in checking buffer space and turning off the sender, but yes,
> it's very critical code and you'd need to find the right place to do that
> work, and that might be difficult.

Usually you should set the handshake when you receive the byte and find too
little space left, i.e. in the IRQ/NMI. After all this is the place where it
fills up.

To make it faster, you only need to _stop_ the PC sending in the NMI. Make the
PC start again you can do in the normal routine that reads from the buffer.
Reading from the buffer is the only place where you free some space in the
buffer, and can thus decide to let the PC send again.

(Note: make sure your code does not have any race condition, like setting the
handshake line from NMI and at the same time resetting from IRQ, as this could
lead to bad results. Usually only change the handshake line when
freespace(buffer) < LOWWATER in the NMI, and when freespace(buffer) >
HIGHWATER in the read routine, where LOWWATER < HIGHWATER.)

Hope this helps (sorry it's late here)

André



>
> Ed
>
>
> On 29 July 2018 at 17:06, Thom Cherryhomes <[hidden email]>
>
> wrote:
> > Yes, I am currently doing this in the main-line code, checking the size of
> > the ring buffer, and if it passes a given threshold, it asserts bit 1 of
> > $DD01, continues to process the now draining buffer, and toggles the line
> > when it's ready for more data...
> >
> > This has worked, up until yesterday, when it's now clear that I am
> > spending too much time plotting text (that's a whole other matter), and I
> > am trying to get back that reliability while I try to either find help to
> > get the text routines fixed, OR ultimately work them out myself...but now
> > I
> > am wondering if moving the buffer check and the handshaking into the NMI
> > handler for the bit banging serial driver will give me the temporary
> > stability I need to make this work?
> >
> > -Thom
> >
> > On Sun, Jul 29, 2018 at 11:00 AM Ed Spittles <[hidden email]>
> >
> > wrote:
> >> To be clear on the directions, then, it's the 8-bit client which is
> >> receiving data, has got too much, or nearly too much, and would like to
> >> tell the upstream host to stop sending?
> >>
> >> I've seen that same situation using the BBC Micro's serial port and
> >> connecting to a PC. The Acorn OS has a buffer threshold parameter, and I
> >> found I needed to adjust it. That is, the BBC needs to maintain a lot of
> >> space in the buffer, and as soon as it has less then 50 bytes it tells
> >> the
> >> PC to stop sending. The reason is that the PC can take its time to
> >> respond
> >> to the request to go quiet, and many more bytes may arrive.
> >> http://beebwiki.mdfs.net/OSBYTE_%26CB
> >>
> >> Others have seen the same, and set the free space to 100 bytes:
> >> https://stardot.org./forums/viewtopic.php?p=58297#p58297
> >> although I see they still report problems.
> >>
> >> It's crucial to have the right cable connections, and a suitably
> >> cooperative serial port behaviour on the PC.  It might even depend on the
> >> responsiveness of the PC OS.
> >>
> >> Ed
> >>
> >>
> >> On 29 July 2018 at 16:45, Thom Cherryhomes <[hidden email]>
> >>
> >> wrote:
> >>> Am barrelling through PLATOTerm, and I've found that checking the
> >>> receive buffer in the main-line code and asserting flow control there
> >>> doesn't seem to be 100% effective in dealing with what still looks like
> >>> some buffer overrun.
> >>>
> >>> Given this code here:
> >>> https://github.com/nanoflite/c64-up2400-cc65
> >>>
> >>> Would adding a buffer overflow check and asserting bit 1 of $DD01 to
> >>> assert hardware flow control work? I'm asking because I know this code
> >>> is
> >>> _very_ timing sensitive, and I am not very familiar with the user port
> >>> serial machinery.



Reply | Threaded
Open this post in threaded view
|

Re: Adding flow control to CC65 up2400 driver.

Thom Cherryhomes
For reference, here's the current code (which does both HW and xon/xoff simultaneously, as the host does xon/xoff, and the hardware can do rts/cts).

/**
 * PLATOTerm64 - A PLATO Terminal for the Commodore 64
 * Based on Steve Peltz's PAD
 *
 * Author: Thomas Cherryhomes <thom.cherryhomes at gmail dot com>
 *
 * io.c - Input/output functions (serial/ethernet) (c64 specific)
 */

#include <cbm.h>
#include <c64.h>
#include <peekpoke.h>
#include <stdint.h>
#include <stdbool.h>
#include <string.h>
#include "../io.h"
#include <serial.h>
#include "../config.h"

extern uint8_t xoff_enabled;
extern ConfigInfo config;
extern uint8_t (*io_serial_buffer_size)(void);
extern void (*io_recv_serial_flow_off)(void);
extern void (*io_recv_serial_flow_on)(void);

void io_recv_serial_flow_off_user_port(void);
void io_recv_serial_flow_on_user_port(void);
uint8_t io_serial_buffer_size_user_port(void);
void io_recv_serial_flow_off_swiftlink(void);
void io_recv_serial_flow_on_swiftlink(void);
uint8_t io_serial_buffer_size_swiftlink(void);

/**
 * io_init_funcptrs() - Set up I/O function pointers
 */
void io_init_funcptrs(void)
{
  POKE(0xD020,0);
  if (strcmp(config.driver_ser,CONFIG_SERIAL_DRIVER_UP2400)==0)
    {
      POKE(0xD020,2);
      io_serial_buffer_size=io_serial_buffer_size_user_port;
      io_recv_serial_flow_off=io_recv_serial_flow_off_user_port;
      io_recv_serial_flow_on=io_recv_serial_flow_on_user_port;
    }
  else if (strcmp(config.driver_ser,CONFIG_SERIAL_DRIVER_SWIFTLINK)==0)
    {
      POKE(0xD020,3);
      io_serial_buffer_size=io_serial_buffer_size_swiftlink;
      io_recv_serial_flow_off=io_recv_serial_flow_off_swiftlink;
      io_recv_serial_flow_on=io_recv_serial_flow_on_swiftlink;
    }
}

/**
 * io_send_byte(b) - Send specified byte out
 */
void io_send_byte(uint8_t b)
{
  ser_put(b);
}


/********* USER PORT *****************************/

/**
 * Return the serial buffer size
 */
uint8_t io_serial_buffer_size_user_port(void)
{
  return PEEK(0x29B)-PEEK(0x29C)&0xff;
}

/**
 * io_recv_serial_flow_off() - Tell modem to stop receiving.
 */
void io_recv_serial_flow_off_user_port(void)
{
  // for now, assume user port.
  POKE(0xD020,0);
  xoff_enabled=true;
  POKE(0xDD01,PEEK(0xDD01)&~0x02);
}

/**
 * io_recv_serial_flow_on() - Tell modem to stop receiving.
 */
void io_recv_serial_flow_on_user_port(void)
{
  // For now, assume user port.
  POKE(0xD020,14);
  xoff_enabled=false;
  POKE(0xDD01,PEEK(0xDD01)|0x02);
}

/************** SWIFTLINK ***********************/

/**
 * Return the serial buffer size
 */
uint8_t io_serial_buffer_size_swiftlink(void)
{
  return PEEK(0xF9)-PEEK(0xF8);
}

/**
 * io_recv_serial_flow_off() - Tell modem to stop receiving.
 */
void io_recv_serial_flow_off_swiftlink(void)
{
  io_send_byte(0x13);
  xoff_enabled=true;
}

/**
 * io_recv_serial_flow_on() - Tell modem to stop receiving.
 */
void io_recv_serial_flow_on_swiftlink(void)
{
  io_send_byte(0x11);
  xoff_enabled=false;
}

-Thom

On Sun, Jul 29, 2018 at 4:25 PM <[hidden email]> wrote:
On Sonntag, 29. Juli 2018 17:42:56 CEST Ed Spittles wrote:
> Oops, I'm in too deep to help there - hopefully I've helped to clarify the
> situation. My guess would be yes, the NMI service routine is a good place
> to act early in checking buffer space and turning off the sender, but yes,
> it's very critical code and you'd need to find the right place to do that
> work, and that might be difficult.

Usually you should set the handshake when you receive the byte and find too
little space left, i.e. in the IRQ/NMI. After all this is the place where it
fills up.

To make it faster, you only need to _stop_ the PC sending in the NMI. Make the
PC start again you can do in the normal routine that reads from the buffer.
Reading from the buffer is the only place where you free some space in the
buffer, and can thus decide to let the PC send again.

(Note: make sure your code does not have any race condition, like setting the
handshake line from NMI and at the same time resetting from IRQ, as this could
lead to bad results. Usually only change the handshake line when
freespace(buffer) < LOWWATER in the NMI, and when freespace(buffer) >
HIGHWATER in the read routine, where LOWWATER < HIGHWATER.)

Hope this helps (sorry it's late here)

André



>
> Ed
>
>
> On 29 July 2018 at 17:06, Thom Cherryhomes <[hidden email]>
>
> wrote:
> > Yes, I am currently doing this in the main-line code, checking the size of
> > the ring buffer, and if it passes a given threshold, it asserts bit 1 of
> > $DD01, continues to process the now draining buffer, and toggles the line
> > when it's ready for more data...
> >
> > This has worked, up until yesterday, when it's now clear that I am
> > spending too much time plotting text (that's a whole other matter), and I
> > am trying to get back that reliability while I try to either find help to
> > get the text routines fixed, OR ultimately work them out myself...but now
> > I
> > am wondering if moving the buffer check and the handshaking into the NMI
> > handler for the bit banging serial driver will give me the temporary
> > stability I need to make this work?
> >
> > -Thom
> >
> > On Sun, Jul 29, 2018 at 11:00 AM Ed Spittles <[hidden email]>
> >
> > wrote:
> >> To be clear on the directions, then, it's the 8-bit client which is
> >> receiving data, has got too much, or nearly too much, and would like to
> >> tell the upstream host to stop sending?
> >>
> >> I've seen that same situation using the BBC Micro's serial port and
> >> connecting to a PC. The Acorn OS has a buffer threshold parameter, and I
> >> found I needed to adjust it. That is, the BBC needs to maintain a lot of
> >> space in the buffer, and as soon as it has less then 50 bytes it tells
> >> the
> >> PC to stop sending. The reason is that the PC can take its time to
> >> respond
> >> to the request to go quiet, and many more bytes may arrive.
> >> http://beebwiki.mdfs.net/OSBYTE_%26CB
> >>
> >> Others have seen the same, and set the free space to 100 bytes:
> >> https://stardot.org./forums/viewtopic.php?p=58297#p58297
> >> although I see they still report problems.
> >>
> >> It's crucial to have the right cable connections, and a suitably
> >> cooperative serial port behaviour on the PC.  It might even depend on the
> >> responsiveness of the PC OS.
> >>
> >> Ed
> >>
> >>
> >> On 29 July 2018 at 16:45, Thom Cherryhomes <[hidden email]>
> >>
> >> wrote:
> >>> Am barrelling through PLATOTerm, and I've found that checking the
> >>> receive buffer in the main-line code and asserting flow control there
> >>> doesn't seem to be 100% effective in dealing with what still looks like
> >>> some buffer overrun.
> >>>
> >>> Given this code here:
> >>> https://github.com/nanoflite/c64-up2400-cc65
> >>>
> >>> Would adding a buffer overflow check and asserting bit 1 of $DD01 to
> >>> assert hardware flow control work? I'm asking because I know this code
> >>> is
> >>> _very_ timing sensitive, and I am not very familiar with the user port
> >>> serial machinery.



smf
Reply | Threaded
Open this post in threaded view
|

Re: Adding flow control to CC65 up2400 driver.

smf
In reply to this post by André Fachat
On 29/07/2018 22:24, [hidden email] wrote:

> Usually you should set the handshake when you receive the byte and find too
> little space left, i.e. in the IRQ/NMI. After all this is the place where it
> fills up.

Yes, the whole point of handshaking is to stop the sender when you're
busy and haven't processed all of the data received so far.

If you only tell them to stop sending when you're not actually busy then
it's not going to work reliably.

> To make it faster, you only need to _stop_ the PC sending in the NMI. Make the
> PC start again you can do in the normal routine that reads from the buffer.
> Reading from the buffer is the only place where you free some space in the
> buffer, and can thus decide to let the PC send again.

It's wasted effort to check for both conditions there.

However as you only need to check at the point when you have actually
received a character,  at 2400 baud you're only able to receive a
character 4 times per frame (2400baud/10 bits/60fps).

The threshold checks should be done in the driver which is written in
assembly & I would make it the drivers responsibility to do all the flow
control, rather than using C callbacks into the application.

The problem will actually get worse with swiftlink, because bytes can
arrive even more quickly.

It was probably done this way to be flexible and avoid duplication in
the drivers, but it's flawed.