FreeRtos Periodic task stops running randomly

Hello All, I am running into an issue where randomly periodic task in my application stops running. I have 5 tasks defined in my system out of which two tasks are periodic i.e. T2 runs every 20msec and T5 runs every 1sec. Rest three tasks are pending on semaphore. Highest priority task T1 is pending on semaphore from ISR. I am getting interrupt every 0.5msec to capture data from two different source. Depending on source of interrupt other two tasks are enabled by giving respective semaphores.
Priority of tasks reduces sequentialy i.e.
T1 -> Highest Priority and T5 -> lowest Priority FreeRtos is configured to run at tick of 10msec. I have made sure below points are enabled – configAssert is enabled, Priority of interrupt is same as what is configured in configMAXSYSCALLINTERRUPTPRIORITY ISR uses xSemaphoreGiveFromISR and portYIELDFROM_ISR CPU load is 60% No stack overflow observed. I was able to reproduce this scenario once in debug mode and found that all other tasks keep running as normal except two periodic tasks which stops executing. I also observed that during this condition xTickCount become greater than xTimeToWake in vTaskDelayUntil() function as a result system might get into blocked state for long time. Even though periodic task have 2nd highest priority, what could cause scheduler to not wake up task after desired time interval ? Also is there a way to kick out of this situation ? I have watchdog enabled in my application. I am feeding watchdog in T5( runs every 1 sec). Since T5 stops running, i was expecting watchdog to kick. But unfortunately, watchdog doesnt trigger which is unusal and other three tasks keep excuting as normal. Any help is much appreciated. Regards

FreeRtos Periodic task stops running randomly

I have 5 tasks defined in my system out of which two tasks are periodic i.e. T2 runs every 20msec and T5 runs every 1sec. Rest three tasks are pending on semaphore. Highest priority task T1 is pending on semaphore from ISR. I am getting interrupt every 0.5msec
That is a relatively high frequency, although not abnormally so. Is the semaphore a counting semaphore? Is it possible to use a task notification (https://www.freertos.org/RTOSTaskNotificationAsCounting_Semaphore.html ) instead as it would be faster? Is you code written to be robust against two interrupts occurring before the task has processed the first?
FreeRtos is configured to run at tick of 10msec. I have made sure below points are enabled – configAssert is enabled, Priority of interrupt is same as what is configured in configMAXSYSCALLINTERRUPTPRIORITY ISR uses xSemaphoreGiveFromISR and portYIELDFROM_ISR CPU load is 60% No stack overflow observed.
Thanks for providing his information without us having to ask first :o) CPU load would seem to indicate 0.5ms interrupt frequency should not be a problem at all – unless you have long critical sections somewhere in your code.
I was able to reproduce this scenario once in debug mode and found that all other tasks keep running as normal except two periodic tasks which stops executing. I also observed that during this condition xTickCount become greater than xTimeToWake in vTaskDelayUntil() function as a result system might get into blocked state for long time.
I’m not sure that is the case. The kernel’s implementation is such that the tick count just being greater than the time to wake is enough for the tasks to be removed from the Blocked state – see the line “if( xConstTickCount >= xNextTaskUnblockTime )” in xTaskIncrementTick(), which is defined in tasks.c. If you are able to replicate this situation in the debugger again then place a break point on that line to try and determine why the tasks are not being unblocked. For example, is xNextTaskUnblockTime wrong in some way? If it is not wrong, then why does it not reflect the unblock time of the tasks?

FreeRtos Periodic task stops running randomly

Richard, Thanks for replying back. To be more precise on my system. we have FPGA which is collecting data from different sensors and provide one interrupt to controller. Alongwitht the interrupt it also provides information on interrupt source. So for now we have two sensors in our system. FPGA generates same interrupt at different intervals based on sensor one is 0.5msec and another one is 0.88msecs. T1 task act as a producer. All other tasks act as a consumer. All other tasks can register their interest on sensor data they need with T1 alongwith required samples. T1 task pends on counting semaphore which then wakes up on interrupt and put data in respective queue’s. Once required sample are collected in their respective Queue’s, I give respective semaphores to wake up T3 and T4 task.. Now xTicksToWait on queue is defined as 0 so that it doesnt block. I decided to use Semaphore alongwith the queue because i wanted to collect x samples prior waking up another task. The semaphores to wake up T3 and T4 tasks are binary semaphores. At the begining i didnt have T3 and T4 tasks, and with the same architecture described above i was logging sensor data in memory. I never had this issue, and the timestamp was sequential which proved that controller didnt miss any interrupts otherwsie i would have seen gap in timestamps. After adding T3 and T4 task i see this issue where periodic task getting blocked forever. Since it blocks T2 task where logging happens i cant see data so i am not sure if system waited long for any semaphores which incremented system tick beyond wake up time. Is this possible ? But even if that is the case since T2 is high priority task, Shoudnt it preempt T3/T4 tasks ? If xTicksToWait in semaphores/queues are made as 0 then they should not block .. Is my understanding correct ? Do you see any concerns in the architecture ? As I might have to add more tasks to the system and pass data between each tasks in future. I havent explored Task Notification yet. But i will give it a try. I will definitely put break point on xTaskIncrementTick() and get back to you with response.

FreeRtos Periodic task stops running randomly

T1 task pends on counting semaphore
Probably not relevant, but using a task notification would be faster…
which then wakes up on interrupt and put data in respective queue’s.
If there is only one reader for each queue, then a message buffer would be faster…
Once required sample are collected in their respective Queue’s, I give respective semaphores to wake up T3 and T4 task.. Now xTicksToWait on queue is defined as 0 so that it doesnt block. I decided to use Semaphore alongwith the queue because i wanted to collect x samples prior waking up another task.
With a message buffer you can effect the same thing by setting a ‘trigger level’.
The semaphores to wake up T3 and T4 tasks are binary semaphores.
Are the tasks structured to drain the queues? Sounds like they are from your description above about having multiple readings in the queues.
At the begining i didnt have T3 and T4 tasks, and with the same architecture described above i was logging sensor data in memory. I never had this issue, and the timestamp was sequential which proved that controller didnt miss any interrupts otherwsie i would have seen gap in timestamps.
Good information.
After adding T3 and T4 task i see this issue where periodic task getting blocked forever. Since it blocks T2 task where logging happens i cant see data so i am not sure if system waited long for any semaphores which incremented system tick beyond wake up time. Is this possible ?
As per my previous post – if the tick goes up to the unblock time the task should be moved out of the blocked state but it may not run immediately.
But even if that is the case since T2 is high priority task, Shoudnt it preempt T3/T4 tasks ?
Yes.
If xTicksToWait in semaphores/queues are made as 0 then they should not block .. Is my understanding correct ?
Yes.
Do you see any concerns in the architecture ? As I might have to add more tasks to the system and pass data between each tasks in future.
Given a CPU load of 60% it should be fine, but the 60% value doesn’t say anything about how bursty the load is – that is what happens during the 40% when the system is busy. In any case, the description you give is not a state that should exist in the first place.
I havent explored Task Notification yet. But i will give it a try. I will definitely put break point on xTaskIncrementTick() and get back to you with response.

FreeRtos Periodic task stops running randomly

Richard, I was able to reproduce the error condition, and i added break point at the if statement as you suggested. I see both below statements get true if(xConstTickCount >= xNextTaskUnblockTime ). if( xConstTickCount < xItemValue ) I could see periodic task getting updated in pxTCB. Eventually in sometime if( xConstTickCount < xItemValue ) statement get false and task is added in ready list by prvAddTaskToReadyList( pxTCB );. I have seen this happening repeatedly but when i let software run periodic tasks doesnt run. Any idea what could not let periodic task run when its added in readylist ?

FreeRtos Periodic task stops running randomly

The only think I could think of would be if the kernel didn’t realise there was anything in that ready list. Do you have configPORTOPTIMISEDTASK_SELECTION set to 1 in FreeRTOSConfig.h?

FreeRtos Periodic task stops running randomly

No.. its set to 0

define configUSEPORTOPTIMISEDTASKSELECTION 0

FreeRtos Periodic task stops running randomly

Richard, I looked into pxDelayedTaskList. I see two issues. I see wait time for 20msec task is not right( it gets bigger value) and when that time comes it doesnt execute the task. Since the tick time is 10msec, i should see nextBlock time in increment of two. However, i see difference between nextBlockTime and current Tick time is 9000-10000 Any idea what could corrupt the blocktime in list ?