FreeRTOS-Plus-TCP goes to HardFault on TCP packet

I’ve started to use FreeRTOS for having multi threading (measure a couple of values simultneously with http and possibly ftp servers). Current target platform is LPC1768 (Landtiger2 board), project developed using CoCoX IDE. Source code here: https://github.com/IzyaSoft/M3WebBackbone/tree/UshakovMV_http (actual branch UshakovMV_http). The problem: FreeRTOS task, tcp stack initialization are ok, and TCP server like in examples starts okay. NetworkInterface based on refactored EasyWeb driver (because it is suitable for my board) works:** i am able to send ping and receive replies**. I am using BufferAllocation2 scheme with heap4.c memory management with 26k of RAM for heap. But when i am sending Http request TCP stack goes to hardfault (according to debug the kast point is prvTaskExitError ). Using tcpdump or wireshark i see that this is occuring on TCP SYN packet.

FreeRTOS-Plus-TCP goes to HardFault on TCP packet

Michael, Your code is releasing a Network Buffer just after sending it to the IP-task. That will most probably cause the hard-fault.
stack goes to hard-fault (according to debug the kast point is prvTaskExitError )
hard-fault’s are difficult to solve. It sounds strange to me that a hard-fault has something to do with prvTaskExitError(), or even that mentioned function is being called. I suppose you have read the documentation about how to debug hard-faults? Sometimes I use a volatile counter for debugging: increasing its value at certain points in the code that are suspect. When the hard-fault comes, at least I see what was the last instruction/function called.
I am using BufferAllocation_2.c scheme with heap4.c memory management
The is a perfect choice.
… with 26k of RAM for heap
Your ipconfigNETWORK_MTU is 1024, that is good. Or even 640 bytes. That will allow to use a longer sliding TCP window. See documentation about FREERTOS_SO_WIN_PROPERTIES on the website. Try tuning the value of ipconfigNUM_NETWORK_BUFFER_DESCRIPTORS, the maximum number of buffers. The function uxGetMinimumFreeNetworkBuffers() will show the lowest number of available buffers ever. In your main.c I read: ~~~~ /* Wait until the network is up before creating the servers. The notification is given from the network event hook. */ ulTaskNotifyTake(pdTRUE, xInitialBlockTime); ~~~~ Let it block for ever (using portMAX_DELAY) and let it unblock immediately when the IP-stack is up and running: ~~~~ void vApplicationIPNetworkEventHook( eIPCallbackEventt eNetworkEvent ) { static BaseTypet xTasksAlreadyCreated = pdFALSE;
/* If the network has just come up...*/
if( eNetworkEvent == eNetworkUp )
{
    /* Wake-up the web server task. */
    xTaskNotifyGive( xWebServerTaskHandle );
}
~~~~ About task priorities:
  • prvEMACTask ( Higher )
  • IP-task
  • Tasks that use the IP-stack such as prvWebServerTask ( Lower )
Other schemes may be used but we normally prefer the above order. ~~~~ for(;;) { /* Wait for the EMAC interrupt to indicate that another packet has been * received. The while() loop is only needed if INCLUDEvTaskSuspend is * set to 0 in FreeRTOSConfig.h. */ /* HT: It would be a bit cheaper to use xTaskNotifyGive() and ulTaskNotifyTake() for this purpose. */ /* HT: No need to test for the return value when using ‘portMAXDELAY’ 🙂 / while(xSemaphoreTake(xEthernetMACRxEventSemaphore, portMAX_DELAY) == pdFALSE); //printf(“Semaphore captured … nr”); / At least one packet has been received. / while(CheckIsDataAvailable()) { //printf(“After index check … nr”); // Obtain the length, minus the CRC. The CRC is four bytes but the length is already minus 1. dataLength = (size_t) CheckAvailableDataSize();//GetReceivedDataSize() – (cRCLength – 1); if(dataLength > 0) { networkBuffer = pxGetNetworkBufferWithDescriptor(0, (TickType_t )0); / HT: Remember that pxGetNetworkBufferWithDescriptor() may fail and return NULL. / / HT: In that case the RX message must still be popped to free the DMA slot. */ networkBuffer->xDataLength = dataLength; rxBuffer._buffer = networkBuffer->pucEthernetBuffer; rxBuffer._bufferCapacity = networkBuffer->xDataLength; Read(&rxBuffer);
            rxEvent.pvData = (void *) networkBuffer;

            // printf("Received data: ");
            // printStringHexSymbols(networkBuffer->pucEthernetBuffer, dataLength, -1);

            // Data was received and stored.  Send a message to the IP task to let it know.
/* HT: You really don’t want to wait in case the queue is full? Not even a few ms. ? / if(xSendEventStructToIPTask(&rxEvent, (TickType_t)0) == pdFAIL) { / HT: Yes the function vReleaseNetworkBufferAndDescriptor() must be called. Message can not be delivered to the IP-task. / //vReleaseNetworkBufferAndDescriptor(networkBuffer); iptraceETHERNET_RX_EVENT_LOST(); } else { / HT: This would be a iptraceNETWORKINTERFACERECEIVE trace point. / iptraceETHERNET_RX_EVENT_LOST(); } / HT: you’re freeing a packet that you’ve just sent to the IP-task. */ vReleaseNetworkBufferAndDescriptor(networkBuffer); iptraceNETWORK_INTERFACE_RECEIVE(); } ~~~~ And some comments about the interrupt-handler: ~~~~ void EthernetIrqHandler() { uint32t interruptCause; /* HT: You need a variable to remember if a task switch is required. */ + BaseTypet xHigherPriorityTaskWoken = pdFALSE;
 while((interruptCause = LPC_EMAC->IntStatus) != 0)
 {
     //printf("Interrupt raised ... nr");
     /* Clear the interrupt. */
     LPC_EMAC->IntClear = interruptCause;

     /* Clear fatal error conditions.  NOTE:  The driver does not clear all
      * errors, only those actually experienced.  For future reference, range
      * errors are not actually errors so can be ignored. */
     if((interruptCause & INT_TX_UNDERRUN) != 0)
         LPC_EMAC->Command |= CR_TX_RES;

     /* Unblock the deferred interrupt handler task if the event was an Rx. */
     if((interruptCause & INT_RX_DONE) != 0)
– xSemaphoreGiveFromISR(xEthernetMACRxEventSemaphore, NULL); + xSemaphoreGiveFromISR(xEthernetMACRxEventSemaphore, &xHigherPriorityTaskWoken); }
 /* ulInterruptCause is used for convenience here.  A context switch is
  * wanted, but coding portEND_SWITCHING_ISR( 1 ) would likely result in a
  * compiler warning. */
/* HT: the parameter of portENDSWITCHINGISR() must be non-zero if a task-switch is required. You don’t know if a switch is required, this depends on the above call xSemaphoreGiveFromISR() */
  • portENDSWITCHINGISR(interruptCause); /* HT: use the new name and the correct variable. */
  • portYIELDFROMISR( xHigherPriorityTaskWoken ); } ~~~~
The task prvEMACTask() is also sometimes called a “deferred interrupt handler”. The interrupt takes place, it registers the interrupt cause and it defers the work to EthernetIrqHandler(). I think that it also must pass TX-done and TX-error events, because you are lending out the Network Buffer to the DMA: ~~~~ BaseTypet xNetworkInterfaceOutput(NetworkBufferDescriptort * const pxNetworkBuffer, BaseTypet xReleaseAfterSend) { BaseTypet result = pdFAIL; struct EthernetBuffer txBuffer; /* Attempt to obtain access to a Tx buffer. / for(uint32_t x = 0; x < MAX_TX_ATTEMPTS; x++) { if( CheckTransmissionAvailable() == 1) //todo add check! { //todo: umv: packetization if( pxNetworkBuffer->xDataLength < ETH_MAX_FLEN ) { txBuffer._buffer = pxNetworkBuffer->pucEthernetBuffer; txBuffer._bufferCapacity = pxNetworkBuffer->xDataLength; txBuffer._storedBytes = pxNetworkBuffer->xDataLength; / HT: Write() a non-blocking function that only passes the buffer to DMA. / Write(&txBuffer); iptraceNETWORK_INTERFACE_TRANSMIT(); result = pdPASS; break; } } / HT: In production version, you could block here on a TX-done event to make it faster. */ else vTaskDelay(TX_CHECK_BUFFER_TIME); }
//vReleaseNetworkBufferAndDescriptor(pxNetworkBuffer);
return pdPASS;
} ~~~~ In the above function, if xReleaseAfterSend is true, you get the ownership of the Network Buffer, and the driver is obliged to release it. You have chosen for : ~~~~

define ipconfigZEROCOPYTX_DRIVER ( 1 )

~~~~ and therefore xReleaseAfterSend will always be true. But remember that vReleaseNetworkBufferAndDescriptor() must be called once the packet has been sent, also in case sending failed. Regards.

FreeRTOS-Plus-TCP goes to HardFault on TCP packet

Thanks, for a lond detailed reply, I am going to check all cases that you have mentioned. Hope, my TCP and of course HTTP finaly will work.

FreeRTOS-Plus-TCP goes to HardFault on TCP packet

Thanks, for a long detailed reply
Welcome. We like to stimulate the development of new drivers. On FreeRTOS you’ll find some drivers posted by other users. Writing a driver for an IP-stack is not easy. Here is a bit more text: One of the difficulties is the ownership of the Network Buffers and the zero-copy behaviour. In xNetworkInterfaceOutput(): ~~~~ BaseTypet xNetworkInterfaceOutput( NetworkBufferDescriptort * const pxBuffer, BaseType_t bReleaseAfterSend ) ~~~~ bReleaseAfterSend tells if the driver becomes the owner of pxBuffer. If bReleaseAfterSend is false, then pxBuffer may only be used within the function. Once the function has returned, pxBuffer will be released or re-used for other purposes. As I wrote, in your driver, where ipconfigZERO_COPY_TX_DRIVER = 1, bReleaseAfterSend will always be true. It allows you to keep using the Network Buffer after returning from xNetworkInterfaceOutput(). When you send a Network Buffer to the IP-task, using the eNetworkRxEvent message, the IP-task will become the new owner. The IP-task will release it when necessary. About the PHY : At this moment your code assumes that the Link Status (LS) is always high. That is normally OK for testing because most PHY’s have workable default settings: use auto-negotiation while advertising all features. The use of hubs is not recommended, it’s better to use switches or routers. Later on, you may want to poll the PHY for its LS: as soon as it drops, it makes no sense to keep on transmitting packets. Note that after powering up your device, the PHY will need a few seconds to set-up the Link. The function xNetworkInterfaceInitialise() is supposed to check for the LS and return pdFALSE as long as the LS is low. It will be called repeatedly until it returns pdTRUE. In the zero-copy driver in portable/Zynq you will see this code: ~~~~~ BaseTypet xNetworkInterfaceOutput( NetworkBufferDescriptort * const pxBuffer, BaseTypet bReleaseAfterSend ) { if( ( ulPHYLinkStatus & BMSRLINKSTATUS ) != 0 ) { iptraceNETWORKINTERFACETRANSMIT(); emacpssend_message( &xEMACpsif, pxBuffer, bReleaseAfterSend ); } else if( bReleaseAfterSend != pdFALSE ) { /* No link. */ vReleaseNetworkBufferAndDescriptor( pxBuffer ); }
    return pdTRUE;
}
/*-----------------------------------------------------------*/
~~~~~ In the same Zynq driver, you will see that several events are passed to the function prvEMACHandlerTask(), using this variable:
volatile uint32_t isr_events;
Hope, my TCP and of course HTTP finally will work.
Yes of course it will 🙂 Once you see TCP connections building up it becomes interesting: the tuning of TCP-buffers and the window sizes. Regards.

FreeRTOS-Plus-TCP goes to HardFault on TCP packet

Hi. I have tryed to change code according to you commentaries, bur Hard Fault still occured, and I found point and seems that reason too: memory-issue Sorry, but image are not displaying, i just post link: https://drive.google.com/file/d/0B9GLCHSwsJn8RF9pNUs0dHJ4eW8/view?usp=sharing According to this image address of pxOwner is 0x5000c4c3, but upper address of my RAM on board is 0x2007C000 + 0x00008000 that is rather smalle than address of pxOwner this code: if(!pxIterator->pvOwner) break; I have added because i see case when pxIterator was not null, but all it fields were. What do you think is it a bug o some bad configuration or maybe it is Cocox fault?

FreeRTOS-Plus-TCP goes to HardFault on TCP packet

Could you please send the adapted files main.c and networkInterface.c ? Or update mentioned files on your github repo ?

FreeRTOS-Plus-TCP goes to HardFault on TCP packet

Done. Branch https://github.com/IzyaSoft/M3WebBackbone/tree/UshakovMV_http

FreeRTOS-Plus-TCP goes to HardFault on TCP packet

Could you please comment what should i fix to eliminate such behaviour or maybe you could give me a direction (unfortunately my current project is quite urgent)?

FreeRTOS-Plus-TCP goes to HardFault on TCP packet

Oops, I had prepared an answer, and I had pressed on Post, but I forgot to check whether my text was accepted. I’m sorry about that. Here some comments on your latest NetworkInterface.c: ~~~~ dataLength = (sizet) CheckAvailableDataSize();//GetReceivedDataSize() – (cRCLength – 1); if(dataLength > 0) { /* The first parameter should be dataLength, not 0. */ – networkBuffer = pxGetNetworkBufferWithDescriptor(0, (TickTypet )xPauseTime); + networkBuffer = pxGetNetworkBufferWithDescriptor(dataLength, (TickTypet )xPauseTime); if(networkBuffer == 0) { vTaskDelay(xPauseTime); continue; } networkBuffer->xDataLength = dataLength; rxBuffer.buffer = networkBuffer->pucEthernetBuffer; rxBuffer._bufferCapacity = networkBuffer->xDataLength; Read(&rxBuffer);
 rxEvent.pvData = (void *) networkBuffer;

 // printf("Received data: ");
 // printStringHexSymbols(networkBuffer->pucEthernetBuffer, dataLength, -1);

 // Data was received and stored.  Send a message to the IP task to let it know.
 if(xSendEventStructToIPTask(&rxEvent, (TickType_t)5*xPauseTime) == pdFAIL)
 {
     vReleaseNetworkBufferAndDescriptor(networkBuffer);
     iptraceETHERNET_RX_EVENT_LOST();
     continue;
 }
 /*else
 {
     iptraceETHERNET_RX_EVENT_LOST();
 }*/
/* HT No delay is necessary here. */ vTaskDelay(5 * xPauseTime); – vReleaseNetworkBufferAndDescriptor(networkBuffer); iptraceNETWORK_INTERFACE_RECEIVE(); ~~~~ The first parameter of pxGetNetworkBufferWithDescriptor() indicates the length of the buffer. In BufferAllocation_1.c, the actual buffer size is fixed at its maximum. Statis buffers are being used. You are using BufferAllocation_2.c, the more economic version. Your code asks for zero bytes length. Some memory will be overwritten that belongs to another task. And as you see, the second vReleaseNetworkBufferAndDescriptor() doesn’t have to be called. You had just transferred the ownership to the IP-task who will release the Network Buffer. Good luck, Hein

FreeRTOS-Plus-TCP goes to HardFault on TCP packet

Thanks a lot, Hein. I am so happy: project stopped to crash and finally i was able to see successfull tcp handshake.