FreeRTOS+TCP (Labs)

I am in the process of creating a small web-server (example) application running on an STM32F4 Discovery board connected to the STM32F4DIS-BB (element14.com) containing a LAN8720 PHY. I got the low-performance implementation going with copying data between buffers. The integrated ping reply works marvelous. While experimenting with the more advanced zero-copy mechanism I have noticed that I cannot find the function “vReleaseNetworkBuffer” anywhere in the tcp-ip code. Am I supposed to implement this myself? If so, is there an example somewhere on how to do that?

FreeRTOS+TCP (Labs)

There are two functions that release memory allocated to a buffer. The first releases both the buffer itself, and the descriptor that points to the buffer. This is called vReleaseNetworkBufferAndDescriptor(). The second releases just the buffer, on the assumption the descriptor that used to point to the buffer has already been released. According to the documentation and your question this should be called vReleaseNetworkBuffer(), but in fact, looking at BufferAllocation2.c, it is actually called vNetworkBufferRelease(). This is a mistake. The naming of the functions used to be consistent, rather than have one use “release” at the beginning of the name and the other at the end – but a decision was made to change their names and (for once) it is the documentation that is correct the function should be called vReleaseNetworkBuffer() as it looks like the source code was not updated accordingly. We now have several target specific examples for the +TCP code and another new component that will be release soon (I keep saying ‘soon’ ;o). At the moment there is only one driver that has been tidied up enough to allow its publication. I have attached this. I’m afraid its not for an STM32, but you may find it useful as a reference all the same. It is not intended as a model implementation, for example it does not handle error conditions properly, etc., but it does work. It also requires the head revision of the +TCP stack – which we can also send you if you like. Regards.

FreeRTOS+TCP (Labs)

Thank you for clarifying, it makes sense now. I would very much be interested in the head revision of the +TCP code. Regarding the STM32 controllers, is there a plan on releasing a target example ‘soon’ :)?

FreeRTOS+TCP (Labs)

Using the Keil MDK-ARM 5 IDE I am getting the following compiler error: tcp-ipFreeRTOS_IP.c(…): error: #513: a value of type “__packed uint16_t *” cannot be assigned to an entity of type “uint16_t *” If I change the declaration line in the function usGenerateProtocolChecksum from uint16_t usChecksum, *pusChecksum; to *__packed uint16_t usChecksum, *pusChecksum;* there is no more error and from what I can tell, the code executes properly. Is this due to the development environment, or am I doing something wrong here?

FreeRTOS+TCP (Labs)

That is an interesting one. A uint16t cannot itself be packed, but in this case it is being used to point to uint16t that is in a packed structure, and I assume the warning is generated because the compiler cannot guarantee that the value being pointed to is aligned. That is, because the structure is packed the uint16_t might end up on an unaligned boundary. I think however, because of the padding added to the beginning of the frame to ensure alignment, the uint16_it will be aligned, but the compiler of course has no way of knowing that. It may be that we have to load the value as two bytes, rather than one 16-bit variable, to prevent the compiler doing this. Hein – do you have anything to add? usGenerateProtocolChecksum() is still needing a bit of a restructure and tidy up so this issue can be resolved one way or another while that is being done. In the mean time your local change will be fine for you – but it is not something we can adopt in the main line as it is Keil specific code. Regards.

FreeRTOS+TCP (Labs)

Well, I am glad, that I’m not the only one a bit confused by this compiler error. Your explanation makes perfect sense, of course my fix is specific for my tool chain. I am sure you will come up with a more portable fix in the future. Thank you and Best Wishes.

FreeRTOS+TCP (Labs)

Hi Pascal, Thanks for reporting. Your compiler warning is correct, as is Richard’s comment 🙂 The risk of assigning a pointer to a packed struct to a normal (unpacked) pointer is to get an alignment exception later on. ~~~~~ struct xPACKET { uint8t a; uint16t b; /* byte-aligned */ uint8_t c; } attribute( (packed) );
struct xPACKET xPacket;
uint16_t *pusChecksum = &( xPacket.b );

/* This assignment might cause an exception. */
*pusChecksum = 0;
~~~~~ Within ‘usGenerateProtocolChecksum()’ we know that the ‘usChecksum’ field in the UDP/TCP/ICMP header is at least 16-bit aligned. I would propose to insert some casts:
    pusChecksum = ( uint16_t * ) ( &( pxProtPack->xUDPPacket.xUDPHeader.usChecksum ) );
    pusChecksum = ( uint16_t * ) ( &( pxProtPack->xTCPPacket.xTCPHeader.usChecksum ) );
    pusChecksum = ( uint16_t * ) ( &( pxProtPack->xICMPPacket.xICMPHeader.usChecksum ) );
Can you try if Keil accepts the cast as a solution? And if not, we’ll have to change the declaration to: ~~~~~ uint16_t usChecksum;

include “packstructstart.h”

uint16_t *pusChecksum  /* without a semicolon */

include “packstructend.h”

~~~~~ “_packed” is simpler but other compilers use a different notations. GCC want this: ~~~~~ uint16_t *pusChecksum attribute( (packed) ); ~~~~~ and for me it warns that the ‘packed’ attribute will be ignored. That’s because my CPU allows 16-bit access at any address. Regards, Hein

FreeRTOS+TCP (Labs)

Hello Hein, Thank you for clarifying further. I have tested your proposed solution with casting. This works fine, Keil is happy, and the code is running. Your alternative (second) solution basically boils down to exactly what I did, as obviously my packstructstart/end files will pre-pend the declaration in question with “__packed”. Regards, Pascal

FreeRTOS+TCP (Labs)

Hallo Pascal,
Your alternative (second) solution basically boils down to exactly what I did
Right, and I thought that the solution with the casting is just a little more esthetic than including the two “packstructxx.h” header files. You are working with a STM32F4… I have just received an evaluation board from ST and I’m about to try-out +TCP myself on this great MCU. Regards, Hein

FreeRTOS+TCP (Labs)

Hello Hein, Oh, that is great! It is an excellent MCU … however it seems I am a bit stuck with getting the zero-copy method going. But that is mostly because I have to learn how the ETH MAC exactly works. It is the first time I am trying to get TCP/IP going on an embedded system. Anyways it would be great if you could give me some pointers on where I’m going wrong. As I promised Richard last week, I will send you my project, what I have so far. Just give me a couple more days to tidy it up a bit. Best, Pascal

FreeRTOS+TCP (Labs)

Hello Hein, I am now adding the final touches to my first TCP/IP project. Got DNS going, have an HTTP server running capable of interpreting shtml and wrote a very simple telnet sever from scratch. To round everything off I’d like to have DHCP working. I set the macro ipconfigUSE_DHCP to 1. It seems to do something, as I can see this entry in my router DHCP table: Unknown 192.168.12.117 02:56:47:00:00:00 55 however, I am unable to ping this IP address. The fall-back IP coded into the firmware doesn’t work either. Also calling the function FreeRTOS_IsNetworkUp() returns pdFALSE. Another thing, why does the expiration time always start at 60 seconds? Any ideas? Thank you in advance, Pascal

FreeRTOS+TCP (Labs)

Is 02:56:47:00:00:00 the MAC address of your target? If so, can you try setting the first octet to 0 and try again (currently it is 2)? If not then… What are you using as a DHCP server? Can you view the DHCP traffic in Wireshark to see how far it gets? If you can see DHCP messages, and changing the MAC address does not help, then please attach a Wireshark log showing the DHCP traffic to your next post. Regards.

FreeRTOS+TCP (Labs)

Hi Pascal, Just to remember, this is a nice DHCP feature: ~~~~~ #if( ipconfigDHCPREGISTERHOSTNAME == 1 ) const char pcApplicationHostnameHook () { return “my_stm”; } #endif / ipconfigDHCPREGISTERHOSTNAME */ ~~~~~ With this code your DHCP server should show:
my_stm 192.168.12.117 02:56:47:00:00:00 55
Same comments as Richard: would it be possible to inspect the DHCP conversation? Those are all broadcasts so you should see them from any laptop. I attached a PCAP in which you see a DHCP conversation. My device is called ‘lpc’.
why does the expiration time always start at 60 seconds?
Doesn’t the DHCP server determine the lease time? If the code receives a lease time of less than 60 seconds, it will use 60 seconds. But normally a lease time will be at least 1 hour. Regards, Hein

FreeRTOS+TCP (Labs)

Thank you so much for the hints, especially the hots-name feature :). my DHCP server is a linksys (cisco) WAG320 modem/router. The lease time is set to 24h, I have reconfigured it to 60min, however the “STM32F407 client still gets an expiration time of 60sec in the DHCP table. I am guessing this is how long the initial offer stands. Somehow it seems to me that my firmware is not able to receive the offer and acknowledge it (see attachment). But it also does not revert back to the hard coded IP. After a while the function vApplicationIPNetworkEventHook detects the link is up and I can see in the debugger that the IP address used is supposed to be 192.168.12.88. Any ping to this address produces a timeout, even though other aspects of the firmware (push-button, LEDs, …) do work properly. Concerning the MAC address, a bit of playing around with it has shown that as long as bit 0 in the most significant byte is not set (unicast not multicast) the client shows up in my router’s DHCP table (for 60 seconds), otherwise it doesn’t. Regards, Pascal

FreeRTOS+TCP (Labs)

Salut Pascal, Thank you for the PCAP. Could it be that your STM32 does not receive any packets at all? Is this firmware able to respond to pings? If not, try without DHCP first :
`#define ipconfigUSE_DHCP    0`
If it can not receive data, you might want to check the GPIO settings of the pins involved, especially for the RX path. Remember that various pins can perform the same function, depending on the hardware design. I just made that mistake, I trusted on the library to set all GPIO’s properly. They were set very properly, but not for my board 🙂 There are about 13 pins that need to get the peripheral ETH function 0x0b and maybe some pin that provide a 25MHZ clock for the PHY. In the function ETH_MACAddressConfig(), I changed one line and I also made the function public: ~~~~~ -static void ETHMACAddressConfig(ETHHandleTypeDef *heth, – uint32_t MacAddr, uint8_t *Addr) +void ETH_MACAddressConfig(ETH_HandleTypeDef *heth, + uint32_t MacAddr, uint8_t *Addr) { uint32_t tmpreg;
 /* Check the parameters */
 assert_param( IS_ETH_MAC_ADDRESS0123( MacAddr ) );

 /* Calculate the selected MAC address high register */
– tmpreg = ( ( uint32t )Addr[ 5 ] << 8) | (uint32t)Addr[ 4 ]; + tmpreg = 0x80000000ul | ( ( uint32t )Addr[ 5 ] << 8) | (uint32t)Addr[ 4 ]; /* Load the selected MAC address high register */ ( * ( __IO uint32_t * ) ( ( uint32_t ) ( ETH_MAC_ADDR_HBASE + MacAddr ) ) ) = tmpreg; ~~~~~ It will set the MO bit (nr 31) which should always be a ‘1’. In the debugger I saw that a particular MAC address did not receive any messages. After setting bit 31, it did receive messages. Correction: for ETH_MACA1HR (2HR and 3HR), it is called:
`Bit 31 AE: Address enable`
Note that if you want to use LLMNR, you have to activate the LLMNR multicast address: ~~~~~

if( ipconfigUSE_LLMNR == 1 )

static const uint8_t llmnr_mac_address[] =
{ 0x01, 0x00, 0x5E, 0x00, 0x00, 0xFC };

/* Program the LLMNR address at index 1.
Index 0 is occupied by the main MAC address. */
ETH_MACAddressConfig( &heth, ETH_MAC_ADDRESS1,
    ( unsigned char *)llmnr_mac_address

endif /* ipconfigUSE_LLMNR */

); ~~~~~ and provide an LMNNR callback function: ~~~~~ BaseTypet xApplicationDNSQueryHook( const char *pcName ) { return strcasecmp( pcName, “STM32F407” ) == 0 || strcasecmp( pcName, “mydevice” ) == 0; } ~~~~~ LLMNR allows you to give your devices logical names, like ‘STM32F407’ or ‘my_device”, any name without a dot. This name will be looked-up by ping, ftp clients and ALL browsers. E.g. I ping my ST-board with this command:
ping my_device
Pinging 192.168.2.111 with 32 bytes of data:
Reply from 192.168.2.111: bytes=32 time=1ms TTL=64
Or surf the the same device on /fr-content-src/uploads/2019/07/index.html Regards, Hein

FreeRTOS+TCP (Labs)

Bonjour Hein, Thank you very much for your elaboration on different configuration mishaps, been there, done that 🙂 . I am very certain, that my board and everything is configured properly. With a static IP I can ping the board, I can show web pages off an SD card, and I can even use my primitive telnet server. There are no worries that the board could not receive and/or transmit data. Sorry that I did not mention this before. However, your explanations on LLMNR are very helpful to me and I will activate this also. But is it necessary for the DHCP server to properly hand out the IP address? Could this be the problem? I always thought it should be possible without LLMNR. So I am back to square one, it seems that basically the setup is running with a static IP, but it cannot, or does not, accept the address assigned to by the DHCP server. Is there anything else to do in the firmware except setting ipconfigUSE_DHCP to 1 ? Am I missing a hook function? But then the compiler should already have a problem, no? Thank you for your time. Sincerely, Pascal

FreeRTOS+TCP (Labs)

So the log does show a DHCP discover followed by a DHCP offer, but the offer does not then solicit a DHCP request, just as if (as per Hein’s post) the offer was never received. I have just taken the offer packet from the Wireshark log and played it through the source code (in the Windows version) and (ignoring the fact that the ID didn’t match) found that the packet was parsed correctly and a DHCP request was sent. If the data received from your DHCP server is parsed correctly then it does seem that either the DHCP offer is not being received as would be expected, or after it has been parsed the resulting DHCP request is not making out onto the wire. The first thing to do would be to see if the DHCP packet is ever being received. Find vDHCPProcess() in FreeRTOS_DHCP.c. There you will see a switch statement that has a case eWaitingOffer. The first line in that case is a call to prvProcessDHCPReplies(). Start by placing a break point on that line to make sure it is reached. If that line is reached find the prvProcessDHCPReplies() function itself. In there you will see it first receives from the DHCP socket, then checks to see if any data was received using if( lBytes > 0 ). Try placing a break point inside that if() statement to see if any data is received. If data is received step through the code to see how the data is parsed. What I am seeing here is that everything is parsed correctly, which sets xReturn to pdPASS. xReturn is then returned from the function, which results in the DHCP request being sent. Regards.

FreeRTOS+TCP (Labs)

Thank you for the debugging hints. The code reaches the first suggested break point just fine, exactly in-sync with my wireshark logging. The second breakpoint is never reached. If I place it on the line if( lBytes > 0 ) the value of lBytes is always -11 (-FREERTOSERRNOEAGAIN or -FREERTOSERRNOEWOULDBLOCK). Using a static IP address (the code is running fine), I have implemented an NTP client, which also uses the function FreeRTOS_recvfrom there everything works fine, the reply is received without any problems. Is it possible that some settings in my FreeRTOSIPConfig.h are wrong, like a timeout not set high enough? Regards, Pascal

FreeRTOS+TCP (Labs)

The read of the socket is not a blocking read, so most of the time -FREERTOSERRNOEWOULDBLOCK would be the expected value, but at some point the data that we know is on the wire because it is seen in the wireshark log should arrive on that socket and it looks like that is not happening. So then the question is – how far does the data get. We know the offer is on the wire, so does it actually get received by the target. If it does get received by the target, how far through the stack does it get before the stack decides not to pass it all the way through to the receiving socket. That is a little more involved to debug… Regards.

FreeRTOS+TCP (Labs)

I understand that this is a bit tricky to debug. I just cannot believe that it is not something very trivial that is missing. Everything else worked right out of the box, so I am certain, that packages, TCP and UDP, get sent and received just fine. I have an NTP client running, which just pulled the UTC time every 60 minutes all night long. Would you have any example using DHCP (tested) on a microcontroller, not the windows simulator, that I could use as a frame of reference to track down the problem? It doesn’t matter if it is not an STM? Regards, Pascal

FreeRTOS+TCP (Labs)

We have it running on several microcontrollers but the core code is the same on all targets, only the port layer is different. Tested with 3 different dhcp servers. Is it possible your MTU is set to small to hold the DHCP packet? Regards.

FreeRTOS+TCP (Labs)

In Pascal’s case, the DHCP reply has around 590 bytes. Could that have been too many? If there is enough RAM, I would set MTU at 1500:
`#define ipconfigNETWORK_MTU ( 1500 )`
The largest message will then be 1514 bytes (14 bytes of ethernet header). The Maximum Segment Size will then be 1460. Remember that if a message is supposed to travel through the Internet, the size must be smaller, use for instance MSS = 1400. This value can be set per socket, by calling FreeRTOS_setsockopt(). There is no need to change ipconfigNETWORK_MTU for this. This week I’m also working with the STM32F407 and +TCP. Regards.

FreeRTOS+TCP (Labs)

… I am a bit stuck with getting the Zero-Copy method going While experimenting with the more advanced Zero-Copy mechanism
A note about TCP, UDP and Zero-Copy: The idea of Zero-Copy is simply said: “Avoid the use of MEMCPY”. The advantage is an increased speed, lower memory use and less power consumption. Disadvantages of Zero-Copy may be an increased complexity and less portability of the source code (non BSD-compatible extensions). When +TCP was being developed (and also earlier while developing +UDP), all possibilities were studied to avoid the use of MEMCPY. These optimisations have been realised:
  • For UDP recv: call FreeRTOS_recvfrom() with the FREERTOS_ZERO_COPY flag and receive a pointer to a network buffer. It must be freed after reception.
  • For UDP send: create a network buffer, fill it, call FreeRTOS_sendto() with the FREERTOS_ZERO_COPY flag, and have it passed straight to the DMA descriptors for transmission.
  • For TCP recv: call FreeRTOS_recv() with the FREERTOS_ZERO_COPY flag and receive a pointer to the internal RX stream. It must be released in a separate call.
  • For TCP send: The Zero-Copy flag is ignored.
  • For NIC recv: the DMA RX descriptors can have memory pointers to (the payloads of) network buffers.
  • For NIC send: the send routine can pass the payload buffers straight to DMA (in stead of first copying their contents.
It was surprising to see how these optimisations work-out on different platforms, boards. The efficiency of using Zero-Copy mostly depends on the type of RAM used. If RAM is on a 32-bit bus, MEMCPY can be (should be) really fast*. I was not able the demonstrate the advantages of Zero-Copy on such boards. If RAM has 16- or 8-bit access, or if the amount of RAM is small, then it is worthwhile to think about using Zero-Copy.
  • The speed of memcpy for 32-bit access: if a CPU has a clock of 100MHz, it can memcpy() between 50 and 150 MByte per second. For 16- or 8-bit memory: memcpy would do 22 MB/sec or less per 100 Mhz clock speed.
On a STM32F0xx running on 168 MHz and using internal SRAM, Zero-Copy is probably not going to bring much performance gain. Copying a full-sized frame will last less than 10 uS. Regards.

FreeRTOS+TCP (Labs)

Thank you for elaborating on the zero-copy mechanism for STM32F4 controllers. Due to my sysclock being 150MHz and the fact that portability is a factor, I have decided to not pursue this any further. Concerning the DHCP issue, it has resolved itself by playing around with some FreeRTOSIPConfig.h settings. Unfortunately I didn’t pinpoint the exact setting which caused the problem, which leads me to suspect that several settings were off. For the record and for other people running into similar issues, my MTU is now set to 1400. DHCP is working, the device is ping-able by host-name and an HTTP server is running as well. I cannot thank you enough for your great support! Regards, Pascal

FreeRTOS+TCP (Labs)

Maybe i just haven’t found it yet, but is there any leaning one way or the other as to how this will eventually be resolved? before we saw this post we played with the “_packed” with our keil compiler for a little while but then just used the attribute((packed)) in the end file since keil is happy enough with that and pre- processor list file looked like the attribute was being implemented OK. i just didn’t want to modify a bunch of code if the next release will be handled differently or if the attribute _((packed)); in the end file is the final solution. thanks

FreeRTOS+TCP (Labs)

Sorry – this is quite a long thread, so I’m not 100% sure which issue you are wanting resolved. If it is something to do with DHCP then we have done quite a lot of development there (and those developments should have been released some weeks ago, but will be soon!) so we can let you know what has changed.