New Ethernet library with improved TCP/IP Stack

[dejan.odabasic]

Hello,

Have you tried using:

Code: Select all

Net_Ethernet_28j60_readPHY(unsigned char reg, unsigned char *h, unsigned char *l) 
Net_Ethernet_28j60_writePHY(unsigned char reg, unsigned short h, unsigned short l)
reg - address
*h - high byte
*l - low byte

Currently there are example to which I could direct you to(regarding GET requests), but I'm guessing that you already solved this.

Best regards.

[borris]

Some examples of GET requests broken down into small parts would be nice. For right now I solved it by increasing the MSS size.

Here is what worked for INT based packets. You no longer have to have Net_Ethernet_28j60_doPacket() in the loop.

First, the datasheet recommends that you wait until the ENC28j60 clock is stable before doing any operations. This does the trick:

Code: Select all

while(Net_Ethernet_28j60_readReg(ESTAT) & 0x01) Delay_ms(200);

Once everything has been initialized (stack, configuration, etc.) then you can enable the interrupts:

Code: Select all

if(mode == SERVER_ENABLE_PKT_IRQ){
          Net_Ethernet_28j60_INT_Direction = 0;    // Set as input
          Net_Ethernet_28j60_INT = 1;              // Enable pullup resistor
          Net_Ethernet_28j60_setBitReg(EIE,0xC0);  // Set Ethernet IRQ for packet ready
     }

Then, in your loop you can poll for a INT on that line, for the AVRPLC16 it's PD3 or INT1.

Code: Select all

// ISR that detects incoming packet request
void ethernet_PKT_ISR() iv IVT_ADDR_INT1 ics ICS_AUTO {
     packet = 1;
}

In your loop:

Code: Select all

 if(packet){
              Net_Ethernet_28j60_doPacket();
              packet = 0;
          }

So far works great for me and has freed up some resources as well as allow me to enter sleep mode.

The real PAIN was that I couldn't see the source for the library and apparently it is turning off the INT on the Net_Ethernet_28j60_Init() function. All changes to the ENC28j60 registries need to take place AFTER the module is initialized.

[borris]

I'll be posting my new webserver on libstock shortly, but wanted to share some of my findings in the meantime:

Interrupt based packet handling is tricky if you have a slow mcu. Currently, I'm running it on a 8Mhz AVR and having some issues with locking. How it works is if there is a packet on the EN24J60 then it will pull its' irq line low. That's great but if you don't handle that irq quick enough (which is how it is cleared), the system is locked in a loop.

Secondly, all sockets are dynamically allocated on my server so that heap is spared if there are no incoming connections. I am also storing all files on USB and / or SD Card media. If you are choosing to do this, I've found that the tx buffer needs to be reduced in size. The library, at default, is 512 bytes. With so much time being spent on reading on the shared bus from SD/USB to tx buffer, you will miss some packets on large web pages. When using dynamic memory this also becomes an issue because there can be cases where the socket is lost midstream and fails to deallocate. I believe I have a pretty good check on that one.

I do have one question:

I'm running this function every 30 seconds to clean up after any disfunctional sockets:

Code: Select all

*/
void webCleanUp(){

    if(dlist_size(&socketList) == 0) 
          return;
     else{
          DListElmt *currentE = socketList.head;
          socket_t* tempS;

          while(currentE != 0){
               tempS = (socket_t*)currentE->payload;
               if(tempS->s->state != 3){
                    dlist_remove(&socketList, currentE);
               }
               currentE = currentE->next;
          }

     }
}

Is the "state" on the socket the best way to look for sockets that may be left open? Or is there another member of the SOCKET_24j60_Dsc struct that is better?

*Note: The dlist_remove function call also calls the Net_Ethernet_24j60_disconnectTCP function before it removes and deallocates the element.

And as for just fun, I transfer a few 100+mb of images from the AVR 8Mhz server to my PC. Only took a few hours!

[borris]

Just another note on IRQ based packet service:

I've been able to utilize sleep modes with this approach and have been successful in dropping power consumption fairly dramatically.

Here is what I am doing to achieve this:

Code: Select all

// Basic setup to handle interrupts from EN28J60
static void webEnableIRQ(){
     Net_Ethernet_28j60_INT_Direction = 0;    // Set as input
     Net_Ethernet_28j60_INT = 1;                 // Enable pullup resistor
     Net_Ethernet_28j60_setBitReg(EIE,0xC3); // Set Ethernet IRQ for packet ready, TX Error, RX Error
     EICRA |= (1<<ISC11);                     // Enable Falling Edges
     EIMSK |= (1<<INT1);                      // Enable external pin IRQ
     EIFR |= (1<<INTF1);                      // Clear IRQ Flag
}

In main loop:

Code: Select all

if(packet){
     webserverReadPacket();
     if((Net_Ethernet_28j60_readReg(0x1C) & 0x40) == 0){
          packet = 0;
     }

And finally the interrupt handling:

Code: Select all

// ISR that detects incoming packet request
void ethernet_PKT_ISR() iv IVT_ADDR_INT1 ics ICS_AUTO {
     char ethStatus;
     // Clear Global IRQ to reset register
     Net_Ethernet_28j60_clearBitReg(0x1b,0x80);
     // Read IRQ status
     ethStatus = Net_Ethernet_28j60_readReg(0x1C);

     if(ethStatus & 0x40){
        packet = 1;
     } else if (ethStatus & 0x01){
          UART_Write_Text("Receive Error\n");
     } else if(ethStatus & 0x02){
         UART_Write_Text("Transmit Error\n");
     }

     Net_Ethernet_28j60_setBitReg(0x1b, 0xC3);
}

Works very well and keeps handling packets until register is empty, then goes to sleep.

[borris]

One additional question about connecting to a remote site:

Trying to connect to a remote webserver and download a small text file. Problem is, when I use a web browser and enter the address everything is fine, but.... doesn't work at all on the Ethernet Library. Here are the steps I take:

Resolve the IP address of the remote server.
Connect
Send a dummy ACK
Process packets waiting for a reply

I'm successful in getting a remote connection however, once I resolve the IP address, nothing is returned. So I tried to put the resolved IP address (which is correct) into the web browser. It askes me for a user name and password. Weird..

So the question is, how can I make a "regular" GET request through the library that would be the same thing as making the request through a browser?

[darko.jola]

Hello boris.

Firs of all, thank you for sharing your experiences with Net_Ethernet library!

I've found that the tx buffer needs to be reduced in size.

I agree with you that for such project would be better if you can change tx buffer size. We can try to implement this sometime.

Is the "state" on the socket the best way to look for sockets that may be left open? Or is there another member of the SOCKET_24j60_Dsc struct that is better?

You can try to use "open" field for this purpose, because it can be happen that "socket" is not in "Conn. established" state (for example state=5, FIN segment sent), but socket is still busy (open = 0).

About your remote server connection question:

I'm successful in getting a remote connection however, once I resolve the IP address, nothing is returned.

Can you tell me did you go to "Net_Ethernet_28j60_UserTCP" function after sending a dummy ACK? If I understand your question correctly, you just need to form proper GET request and put into in Tx buffer (inside "Net_Ethernet_28j60_UserTcp").

Thanks again.
Best regards
Darko

[borris]

You can try to use "open" field for this purpose, because it can be happen that "socket" is not in "Conn. established" state (for example state=5, FIN segment sent), but socket is still busy (open = 0).

Looking at an open condition seems to be working, thanks.

Can you tell me did you go to "Net_Ethernet_28j60_UserTCP" function after sending a dummy ACK? If I understand your question correctly, you just need to form proper GET request and put into in Tx buffer (inside "Net_Ethernet_28j60_UserTcp").

I'm doing everything correct, I believe it has something to do with the port of the site I'm trying to obtain data from.

I do need some advice on getting larger than the windows requests. I have the :

Code: Select all

onst unsigned int MY_MSS_28j60 = 100;           // Our maximum segment size.

windows set to 100 bytes, but on a particularly large form I'm serving, has 400 bytes on the submit. How can I break up the request into multiples? You had volunteered an example, but haven't seen one yet. Any chance of getting one?

I'll share with the world my function for retrieving the request:

Code: Select all

static DListElmt *initSocket(SOCKET_28j60_Dsc *socket)
{
    socket_t *currentS;
    char termination = 0;
    char request[REQUESTLIM+1];

    int i = 0;
    
    if(dlist_size(&socketList) >= NUM_OF_SOCKET_28j60-1)  // See if there is room in the list
        return 0;

    // Get request, terminate after 2 \n char 
    do {
        request[i] = Net_Ethernet_28j60_getByte();

        if(request[i] == '\n')
            termination++;

        i++;
    } while(termination < 2 && i < (REQUESTLIM));
    
    request[i] = '\0';   // Add a terminating null to string
    
    if(memcmp(request, "GET", 3) != 0 && (socket->state != 3))  // !GET request or state !3  return
        return 0;
   
    if((currentS = (socket_t*)Malloc(sizeof(socket_t))) == 0)  // Lets try to allocate some memory
        return 0;

    strcpy(currentS->requests.rawRequest, request); // Copy request onto socket struct
    currentS->s = socket;                           // Assign socket
    currentS->socketSM = SM_HTTP_PARSE_REQUEST;     // Set the state machine to ready for parsing
    dlist_ins_prev(&socketList, dlist_head(&socketList), currentS);       // Adds element to the head of the list

    return socketList.find(socket);
}

Alot of it won't make sense since I'm using some unlisted structs and a linked list to store all my active sockets in, but you can see from the do / while loop, that is where I am reading the request. I terminate the loop when I have a complete 2 lines or limit is reached.

[darko.jola]

Hello,
when you call Net_Ethernet_28j60_getByte() in your loop, you should take care about TCP payload size (dataLength field of SOCKET_28j60_Dsc). When you read all of payload bytes from received packet, you must wait for next call Net_Ethernet_28j60_UserTCP to read next packet (which contain rest of request).

Best regards
Darko

[borris]

Have another issue that is making me loose some hair:

The following instructions run perfectly on AVR

Code: Select all

while(Net_Ethernet_28j60_readReg(ESTAT) & 0x01) {
        debug_print("Waiting for Ethernet\n\r");
        Delay_ms(200);
    }

When running the same call on a STM32 ARM it never finds the clock for the 28j60 and does not exit the while loop.

Any ideas why this might be?

[darko.jola]

Hello.
You should wait in loop until CLKRDY becomes 1 (OST has expired and PHY is ready), not 0 according to your code.

Take a look into ENC28J60 datasheet:

CLKRDY: Clock Ready bit
1 = OST has expired; PHY is ready
0 = OST is still counting; PHY is not ready

Best regards
Darko

[borris]

Perfect. Thanks!

Wondering though, why does the code work on AVR but not on ARM? Weird.

[darko.jola]

For AVR it can happen because you enter the loop during OST and immediately exit the loop.

Best regards
Darko

[borris]

I understand that Net_Ethernet_28j60_doPacket() cannot be re-entrant. I have, however, found the performance to be AWESOME running it inside an ISR.

Code: Select all

Net_Ethernet_28j60_clearBitReg(EIE,INTIE);      // Clear Global IRQ to reset register

    ethStatus = Net_Ethernet_28j60_readReg(EIR);    // Read IRQ status
    
    if(ethStatus & PKTIF) {
        Net_Ethernet_28j60_doPacket();
    } else if((ethStatus & TXERIF) || (ethStatus & RXERIF)) {
        Net_Ethernet_28j60_clearBitReg(EIR, TXERIF);
        Net_Ethernet_28j60_clearBitReg(EIR, TXERIF);
    }
    
    Net_Ethernet_28j60_setBitReg(EIE, INTIE);

Does anyone know if this could be problematic?

I hope not. The page load times are 10x faster with this as apposed to polling or having doPacket() in the while loop.

[borris]

Hello,
when you call Net_Ethernet_28j60_getByte() in your loop, you should take care about TCP payload size (dataLength field of SOCKET_28j60_Dsc). When you read all of payload bytes from received packet, you must wait for next call Net_Ethernet_28j60_UserTCP to read next packet (which contain rest of request).
Darko

Here is the issue:

The size of the dataLength is ALWAYS the same size as MY_MSS_Intern, so how do I determine the packet total size?

[darko.jola]

Field dataLength contain payload size of TCP packet, and it should not be always equal to MY_MSS. If remote side sends data over multiple packets, it tries to send max bytes of data over each packet. So in most cases size of each packets, except last, is MY_MSS bytes (maximum size of bytes that we can receive in one packet).

Best regards
Darko