Threads, Mutex and Wait-Notify

1. Approach

Multithreading is used usual if the platform has an operation system.

For simple interrupt / mainloop programming see Intr_vsRTOS.html: Program and Process in Interrupt and Backloop for Embedded]

Sometimes instead lock mutex and wait/notify also an atomic access can be used, see Atomic_emC.html: Lockfree programming with atomic access]

Last not leas event handling is sometimes favored, see todo

If more threads runs independently, and the threads should exchange or mutial access data, two mechanism are necessary:

mutial exclusion of access (often in the past 'Semaphores' are used for that)
wait for data, notify to a waiting thread.

Both mechanism have sometimes similar approaches (data exchange). The wait-notify is an alternative to polling or event handling.

The approach in emC is, offer features that are truly resilient and runnable in all environments and situations. Whereby, focused on the real necessary functions and not on a lot of nice to have or possible variations. The role model here is Java which its simple and proven functionality for multi threading.

There are a lot of OS-specific solutions (POSIX with pthread, MS-Windows with a lot of possibilities) and also a common offer in C++: one link of them: https://en.cppreference.com/w/cpp/thread/timed_mutex/try_lock

If you read the text there "This function is allowed to fail spuriously and return false even if the mutex is not currently locked by any other thread." and "If try_lock is called by a thread that already owns the mutex, the behavior is undefined." then it means: "The show must go on" to search for possible solutions in special situations, this functionality seems not to be proven.

2. Approaches of mutual exclusion access for consistent data

If one thread calculates data, and the other thread want to use it, the data should be consistent for usage.

2.1. Simple example get the minute and hour of time while the hour expires

A simple example: If you want to know the time, you should know the hour and the minutes. That are two information. Assume, it is "10:59", you ask for the minute, you get "59". Then meanwhile (in the other thread) one minute expires, the time is "11:00". Secondly you ask to the hour, you get "11" because it is "11:00". Just, the information what you have is "11:59". And that is false, result of inconsistent data.

In this simple example there is a simple solution: Ask the minute, then the hour, then the minute again. If the minute is changed to a lesser value, then ask the hour again. This works because of the special conditions how both values are depending, and how they are changed. But it is not an universal solution.

A proper solution without mutex is event handling: You inform the clock thread to get the time. The clock thread prepares the information, for example "10:59" and sends it to the event queue. Then, (the clock thread works serial for itself), it may increment the time. In your event queue you get a consistent information: "10:59". Of course you get this information in real time a little bit later, and the time is meanwhile "11:00", but this is not the topic of mutex, it is a topic of real time processing. The quest of the clock is only an example here.

2.2. The solution with mutex is done in the following form:

All accesses to data which should be consistent and used by several threads are wrappend with a lock and unlock of a mutex object:

lock data
get the data
unlock data

and

lock data
set or change the data
unlock data

This accesses can be done in several threads. One thread may access the time, the other thread sets a new time. If one thread starts its access with lock dataForClock and the other thread want to do the same lock dataForClock because should also access, then the second, later thread should wait (is displaced from working) till the other thread unlocks unlock dataForClock. Hence the data are never accessed from anywhere other during this locking phase. The data are consistent if another thread accesses.

2.3. Special case: read access often without mutex, mutex and wait/notify if another thread writes yet

There may be a varied approach: Don’t lock only for reading data, because saving time on frequently reading operations in several threads and rarely write operations. This can be solved by a two flags and wait/notify:

ctRead += 1;       // atomic operation
if(flagwrite) {
  lock;            // lock and wait only if anyone writes.
  read Access;     // access under lock till end of read
  ctRead -=1;
  unlock;
} else {           // after increment ctRead a writer does not access
  read Access;     // hence, in most cases fast access without lock
  ctRead -=1;
}
if(ctRead == 0 && flagWrite) {
  lock
  if(ctRead ==0  && flagWrite) {
    notify
  }
  unlock
}

The reading process increments only a numeric value, can be done as atomic access, does not need a mutex, it is fast. Only if a writing process waits, it should be notified. This is quest firstly without the lock (save time) to check its necessity. Only notifying is really necessary, it is quest again and executed under lock. Quest again is necessary because another thread may execute the notify already, accessed before with ctRead +=1 and all, per accident in the short time between quest and lock.

The writing thread do:

lock;
flagwrite = 1;     // annouce, attemp to write
                   // up to now a reading process waits.
if(ctRead >0) {    // reading is interrupted, thats why
  wait();          // wait for notify on end of read.
}                  // any newly ctRead increment is not possible because flagwrite and mutex
write Access       // read thread detects flagwrite, hence it waits in mutex
flagwrite = 0;
unlock;            // From now on reading without mutex can be executed again

The writer waits if anyone is in reading. Because of the flagwrite the reading process will be notifying. Because of the flagwrite a new reader waits in lock. After write access all is normally.

This solution is more complicated in programming but saves time for most accesses. It needs the Approaches of wait/notify (wait/signal), but is elsewhere implementable with the here given solution.

2.4. OS-independent declarations for Mutex in emC

This declarations are contained in src_emC/emC/OSAL/sync_OSemC.h. This file starts with:

Cpp: Mutex type in sync_OSemC:

#ifndef DEF_Mutex_OSemC            //maybe specific defintions may be existing. Defined in compl_adaption.h or via applstdef_emC.h
typedef struct Mutex_OSemC_T {
  const char* name;
  /**This refers OS-internal data for MUTEX allocated on createMutex and removed on deleteMutex_OSemC.
   * The internal type for pthread usage is defined in os_internals.h,
   * included only in the implementation file (os_mutex.h)
   * because OS-specific files should not be part of the user sources.
   */
  void* osHandleMutex;

  /**null then the mutex is not locked. Else: handle of the locking thread. */
  struct Thread_OSemC_T const* lockingThread;

  /**Number of lock calls of the mutex in the SAME thread. Reentrant lock should be supported. */
  int32 ctLock;

  /**Time of the last access.*/
  int32 millisecLock;
} Mutex_OSemC_s;
#endif //DEF_Mutex_OSemC

This definition regards, that the type for Mutex can be defined also in another way, for example especially for a simple Interrupt/Mainloop solution for embedded control without RTOS it is defined as

Cpp: Mutex type in specific_OSemc.h for simple embedded

#define DEF_Mutex_OSemC  // do not use the common definition.
typedef struct Mutex_OSemC_T{
  int16 maskIntr;   //:bits which system-interrupt should be disabled.
}Mutex_OSemC_s;

Also for an embedded program using only Interrupts and Mainloop (see Intr_vsRTOS.html) a Mutex is sensible, because data evaluated in the backloop may elsewhere inconsistent if an interrupt comes and changes the data during access in the main loop. That is able to prevent by "disable interrupt" as machine instruction possible for that level. It is intrinsic the same approach as Mutex mutial exclusion. The user program (any resued algorithm in the main loop) should not distinguish in which environment it runs. The given and necessary mutexes are able to manage (set the interrupt mask) in a specific startup part.

The following operations work with mutexes defined in the src_emC(emC/OSAL/sync_OSemC.h:

Cpp: create a Mutex

#ifndef  lockMutex_OSemC 
//Note: for simple processors without multithreading but with interrupt this identifier may define as macro in the compl_adaption.h or in the applstdef_emC.h
//Then it are not defined here.

/**Creates a mutex object on OS-level.
 * @param thiz reference to data space, can be statically, or also allocated.
 *    It will be cleaned here (set to 0 before usage).
 *    The struct type is usual OS-specific. It contains the necessary data for example a HANDLE etc.
 * @param name Name of the Mutex Object. 
 *  In some operation systems this name should be unique.
 */
extern_C int createMutex_OSemC ( struct Mutex_OSemC_T* thiz, char const* name);

As you see here, all prototype definitions are garded with a macro detection. The macro for lockMutex_OSemC may be defined in a target specific file (<os_types_def.h> just for embedded control for a fast access with __asm, not using a function call.

This operation should create a proper Mutex Object on OS-Level and fill the given structure. After them the mutex can be used.

Cpp: lock and unlock a Mutex

/**locks a mutex. 
 * The contract is: 
 * * If the same thread tries to lock a mutex again, it is okay. It is reenterable for the same thread.
 * * Another thread waits until the owner thread calls unlockMutex_OSemC(...).
 * A timeout is not supported. Timeouts are only a result of programming errors which causes deadlocks.
 * This should be able to found by debugging and monitoring all mutexes.
 */
extern_C bool lockMutex_OSemC(struct Mutex_OSemC_T* thiz);

/**Unlocks the mutex. It is possible that a thread switch occurs, 
 * if another thread waits and it has a higher priority. 
 * In some implementations it is possible that the thread is checked. 
 * The same thread which calls lockMutex_OSemC(...) should call unlockMutex_OSemC(...).
 * If another thread unlocks, it is an error and an exception may be thrown.
 */
extern_C bool unlockMutex_OSemC(struct Mutex_OSemC_T* thiz);
#endif

The header file contains the contract in the comment. The lock should be re-enterable for the same thread, see chapter recursive locking the mutex (lock more as one time) in the same thread.

On unlocking it may be checked that the same thread unlocks (expected). This check can force an exception to detect programming failures.

Cpp: delete a Mutex

/**Deletes the OS mutex object.
 * @param thiz reference to data space as given also by the other mutex operations.
 *   After deletion of the OS-specific Mutex the handle is set to null here also.
 */
extern_C int deleteMutex_OSemC(struct Mutex_OSemC_T* thiz);

Of course a Mutex at OS-Level should be deleted if it is no more necessary. Usual a mutex remains so long the process runs. On most Operation systems Mutexes will be deleted on end of process by the OS. So this operation seems to be lesser necessary.

For implementations of this operations see chapter Implementation of the OS routines for Windows, Linux and simple embedded control with Interrupt and Backloop

2.5. Accidence of errors on programming mistakes

The mutex object should be anytime related to the data which should be consistent. If different programmer do their work in different times, sometimes not all appropriate data are regarded, and sometimes different mutex objects may be erroneously used. That are errors in the software.

The problem for test and detection is: Often an error is not obviously, it is not detected by test. If you access the time, the access needs only 2 seconds between looking on minute and hour, the accident to get a faulty inconsistent time is ~ 1/3600, because only in one of 3600 seconds the minute changes and the hour changes also. If you accesses the time in round about 30 minutes distance, this error comes after ~ 73 days. If your test time of this problem is only one day, you might say, all is okay without mutex. But your application may run 10 years. It means on ~5 days in the year you get a timing error of one hour. Your customer may remark the error at the 4^th occurrence, but on the 5^th occurrence the period of guarantee of your solution is expired, any you are free. But you have written a bad software and the Saint Peter in the software heaven does not give Karma points for you.

It means, mutex problems are often not obviously in test. It should be resolved by a proper software architecture.

2.6. recursive locking the mutex (lock more as one time) in the same thread

Presumed, you have an operation which sets new data, and of course this operation use a mutex:

void operationSetData() {
  lock mutex
  set data
  unlock mutex
}

Now we know that derivation (inheritance) is a proper principle in the Object Orientated Programming. One have a base (super) class, enhances it with more data, and work with it. The second known principle for this topic is: "don’t repeat yourself", means you should not write an special solution for a problem, which is solved already in a proper kind. It means the operationSetData() should be reused also in a derived class:

void derivedOperationSetData() {
  lock mutex
  set some more data
  operationSetData();   // use of course the opeartion from the super class
  unlock mutex

This should work. And it works in Java, and also in MS-Windows

https://stackoverflow.com/questions/2821263/lock-a-mutex-multiple-times-in-the-same-thread

https://stackoverflow.com/questions/12970299/recursive-synchronization

https://learn.microsoft.com/en-us/windows/win32/api/synchapi/nf-synchapi-entercriticalsection

"After a thread has ownership of a critical section, it can make additional calls to EnterCriticalSection or TryEnterCriticalSection without blocking its execution. This prevents a thread from deadlocking itself while waiting for a critical section that it already owns. The thread enters the critical section each time EnterCriticalSection and TryEnterCriticalSection succeed. A thread must call LeaveCriticalSection once for each time that it entered the critical section."

But for the pthread solution (POSIX standard) the recursive locking seems to be not supported by all. My experience is given for Cygwin, and there it is not supported. In the years where POSIX was established as common standard for UNIX systems (in the 1980^th) , the recursive lock for mutex may not be in focus, and hence it was not defined. Java in the beginning of 1990^th has defined it, it was known, it was used, and hence Microsoft’s development in the second part of 1995^th have used this knowledge, have also support recursive lock. On POSIX the programmers knows what they do. They have there own solutions, and all push to bring it to standard were no more successfully. The programmers do what they think, are not interesting to subordinate to any new standards. This is a similar constellation as in defining the programming language C (and also C++) - used by a big community with non-ready clarified standard, the standardization comes too late.

There may be a second reason for non supporting recursive locks: It needs additional calculation time and it is not used in many cases. Hence developer says "Why we should spend calculation time for a thing which is not necessary". But this is a short thinking by developers. The software architect says: "The solution should be reusable in all circumstates". And another practical approach says: "Usual, if anyway mutex is used, it is not for very fast calculations". There is another solution for many problems instead using a mutex: Atomic access. Atomic_emC.html: Lockfree programming with atomic access Java has introduced the package java.util.concurrent.atomic with the version 5 in ~ 2004. This was, because Java runs on servers, which should be fast interact. The atomic access is possible in many specific cases, saves calculation time and it is also possible for embedded control using in interrupts.

The definition of mutex in emC follows firstly the Java approach, which is also supported by MS-Windows. Hence the Linux pthread solution have a special implementation to support this (with a little more calculation effort) without necessity of specific "non portable" enhancements of POSIX pthread.

2.7. Timeout for mutex

Some operation systems supports a timeout of locking a mutex, especially POSIX with pthread: pthread_mutex_timedlock(…) and also in MS-Windows on using WaitForSingleObject(…, timeout) with a CreateMutex(…) as handle. MS-Windows does not support timeout on the simple and fast CRITICAL_SECTION construct. But Java does also not support timeout in its synchronized basic construct. There are possibilities for debugging deadlocks in the JDK, but this is not programmed in the software. In Java there are some extensions with the package java.util.concurrent.locks. A proper description is https://winterbe.com/posts/2015/04/30/java8-concurrency-tutorial-synchronized-locks-examples. But that are also sophisticated enhancements.

Usual the time locking a mutex to do some non interruptable data accesses should be so short as possible. Only the data accesses itself should be done under mutex , and not the preparation of the data with maybe longer operations, mutex access to other data (inside called operations, not obviously, on file accesses etc). This should be the style guide for programming.

Hence a mutex is locked anytime only for a short time, else: In the locking phase a higher prior thread may interrupt a locking lower prior interrupt. And this higher prior interrupt does anything a longer time, independent of the mutex. Then the mutex is locked for a longer time.

But if that mutex is need by another (e.g. higher prior) thread, the chapter priority inheritance for waiting on mutex does the work. The locking lower prior thread gets while the locking phase the necessary higher priority, finishes it works, and also in this case a timeout for lock request is unnecessary.

The only one situation where a timeout is necessary for locking is: In a deadlock situation.

The deadlock situation should not be solved by programming constructs (react on deadlock with timeout, does anything other), it should be prevented fundamentally by a proven programming style. This needs detection, debugging and thinking about. In a running application any thread may wait for success, but usual in a wait/notify construct, this have a timeout. If that timeout comes, maybe some threads should be aborted and restarted.

It is also possible to have one monitor thread, which looks into all mutex objects to detect how long they are locked, whether they seems to have a deadlock, and also the frequency of access. If this monitor thread detects a problem, it should not be fixed by any software action, it should be logged and reported. Longer locked mutexes and deadlocks have its cause in faulty programming. That should be solved. The only sensible reaction may be: Abort a process and restart ist, but only if any other failure comes additional.

Planning a timeout for lock mutex is an error-prone programming style. Hence, though supported by some Operation System specific constructs, it is removed as part of the emC suite.

If a timeout is necessary, for example to find the reason of a deadlock, the user should use temporary specific operations in the used OS. It should not be substantiated in portable reusable sources.

2.8. priority inheritance for waiting on mutex

It is possible that a low prior thread locks a mutex, and the suddenly per accident, it is interrupted by a higher prior (middle prio) thread. It means the mutex remain locked for a longer time.

But now a high prior thread accesses the mutex object, it should wait because the mutex is locked. The really high prior thread should now wait for freeing the mutex, but firstly, maybe, the middle prior thread is continued, and this thread works and works. It means the high prior thread is stopped for a longer time. This scenario is denoted as "inversion of priority", see article https://en.wikipedia.org/wiki/Priority_inversion.

Is that correct ?? The answer is "no" of course. Thats why usual the "priority inheritance" is implemented usually by the scheduler, see also https://en.wikipedia.org/wiki/Priority_inheritance.

The higher priority of the waiting thread is temporary assigned to that thread which locks the mutex with lower priority. It means, now for the example, the lower prior thread continues, because is is now higher prior than the middle prior thread, finishs it work under mutex and unlock the mutex. But on unlocking the priority is assigned as before, it means after unlocking the lower prior thread has its low priority back again, and hence the high prior thread can enter the critical section to access the data under mutex, and continue afterwards.

This principle does work in any case if it is correct implemented in the operation system and and the user programming does not play malicious games with mutexes.

3. Approaches of wait/notify (wait/signal)

This is a little bit similar event processing, or more exact: event processing uses internally wait-notify. A thread waits for any event from another process. The event is here only a signal (in Java denoted as notify which may be a more concise word). The thread gets the data afterwards, or better immediately in the given mutex which surrounds the wait().

For event processing, the event execution thread awaits events (messages) in a queue, the notifying thread fills the queue (a data container) and notifies the waiting event thread, which gets the message out of the queue and executes it as event.

3.1. Principle

A thread should wait for specific data. Hence the thread checks the data, and calls wait() if the data are not available or not in the expected state.

Another thread prepares the data. This other thread should either know that the other one thread waits for new data, or this thread should make a broadcast information about the new data. The last one is a special form, in exactly thinking.

With the notification the waiting thread is awaken and can now access (or check) the data.

The simple form seems to be:

//thread1:
while(data not proper) {
  wait(waitNotifyObject)
}
access the data

//thread2:
changes the data
if(waitNotifyObject !=null) { // is their anybody does waiting?
  notify(waitNotifyObject);
}

On running time, the waiting thread may have a higher priority. It means the waiting thread continues with working because of the notify, and the notifying thread is displaced for this moment, till the thread1 waits for another reason or just again on the same one.

3.2. Wait and notify should check data under mutex

But this simple solution has some pitfalls inclusively deadlock:

What about, if the thread1 is started a little bit later, the thread2 has already notified for new data, do further actions. And now the thread1 comes in the data quest situation. It calls wait(…) but is never notified, because the thread2 has notified in the past already. The thread2 by itself now waits for data from thread1, but thread1 comes never to notify, it waits wrongly for thread1. Wow, we have a deadlock.

This is a little bit mismatch of timing, but maybe bad for the running of the system. This scenario may occur only in specific situations, not seen by tests, not obviously in the guarantee time. And again, you have bad Karma points not only from Saint Peter, also from your customers. But in this case not only inconsistent data occurs, a deadlock occurs.

Thats why exact state information should be given for data and wait conditions. Because this state informations should be accessed with mutual exclusion, it is recommended to wrap it with mutual exclusion locks and unlocks:

//thread1:
while(data not proper) {
  lock(mutexForWaitNotify);
    if(data not proper) {
      flagwait = true;
      wait(waitNotifyObject, mutexForWaitNotify);
      flatwait = false;
    }
  unlock(mutexForWaitNotify);
} //while... data now proper
access the data

//thread2:
changes the data
  lock(mutexForWaitNotify);
    if(flagwait) {
      notify(waitNotifyObject, mutexForWaitNotify);
    }
    set data proper
  unlock (mutexForWaitNotify);
}

The wait(…) is called under mutex, and also the notify(…). Because the threads should not be in deadlock (because one locks and the other needs the lock to maneuver the other out of the lock situation), it is necessary that wait(…) and notify(…) unlocks also the specific lock. Thats why the mutex object is also given as argument for wait(…) and notify(…).

This is necessary to write a stable solution. This is supported by operation systems:

In Java the compiler requires a synchronized, elsewhere it is a compiler error. You can use the following pattern which the data:

Java: wait with mutex:

boolean dataPrepared;           // local variable
synchronized(data) {            // under mutex (!):
  if(!data.prepared) {          // check whether data are already prepared
    data.wait = true;           // notice, anybody waits.
    data.wait();                // wait only if not. The mutex is free inside wait()
    data.wait = false;          // mutex is locked again
  }
  dataPrepared = data.prepared; //access data under mutex
  if(dataPrepared) {
    data.get();                 // get data under mutext
  }
} // end of synchronized block
if(dataPrepared) {              // it will be possible that wait is end without data prepared
  //...evaluate                 // evaluate the local copy of the data without mutex.

Java: wait with mutex:

synchronized(data) {
  prepareData();
  data.prepared = true;         // notice data are prepared
  if(data.wait) {
    data.notify();              // notify only if necessary
  }
} // end of synchronized block

And also the pthreads knows the mutex on wait:

Cpp: pthread_cond_wait uses a mutex argument:

//pthread.h
int pthread_cond_wait (pthread_cond_t *, pthread_mutex_t *);

With the mutex argument the cond_wait unlocks the mutex inside the thread scheduler while wait. This is a atomic operation inside the scheduler, hence safe.

See https://linux.die.net/man/3/pthread_cond_wait

Excerp from this side:

"These functions atomically release mutex and cause the calling thread to block on the condition variable cond; atomically here means "atomically with respect to access by another thread to the mutex and then the condition variable"."

In windows this is also supported by: https://learn.microsoft.com/en-us/windows/win32/api/synchapi/nf-synchapi-sleepconditionvariablecs:

Cpp: SleepConditionVariableCS uses a mutex argument:

BOOL SleepConditionVariableCS(
  [in, out] PCONDITION_VARIABLE ConditionVariable,
  [in, out] PCRITICAL_SECTION   CriticalSection,
  [in]      DWORD               dwMilliseconds
);

But other functions in the windows-API as WaitForSingleObject https://learn.microsoft.com/en-us/windows/win32/api/synchapi/nf-synchapi-waitforsingleobject does not so. Maybe this is also a learning process for windows systems.

3.3. Notify all

It is possible that more as one thread waits for the same waitNotify Object. a simple notify awakes only one thread (which …. the highest prior, depends from OS).

notifyAll awakes all threads. But then only one (the more prior) gets the mutex, the other threads waits for the mutex. The prior Thread does its access. Maybe it sets the data as invalid again (because it has gotten the data). If the other threads get the mutex, they check the situation under mutex, and maybe, data are meanwhile no more accessible. That may the expected situation and programming approach. It is ok.

Usual always notifyAll() can be used. It needs more time in the thread scheduler because all threads are awaken and the most threads should be waiting again after them.

The emC OSAL adaption supports both, as usual all operation systems.

4. The old solutions with semaphores

In the 1980^th multi threading was familiar. Usual the programmer uses semaphore mechanism. A semaphore is set or reset with a atomic access (no interruption between this action). A thread waits if a semaphore is not free. There are also some sophisticated solutions with counting semaphore etc.

This thinking is present sometimes yet. But it seems to be that the principles mutex and wait/notify (or wait/signal) are more used in newer solutions. Working immediately with semaphores is general similar but more sophisticated.

5. History and Comparison to Java

In the 1980^th the multithreading was also familiar for process control computer. But the embedded control was in development, using this approaches. In the beginning of the 1990^th the most used Operation system MS-DOS has no thread support, it was very simple. In opposite to that the development of Java as universal language supporting different processor platforms (with the byte code approach) has introduced a proper system for Threading. It has used the full experience of the 1980^th. In Java, both the mutex access (mutial exclusion) and the wait/notify is very simple and powerful supported.

In Java the base class of all, java.lang.Object contains the capability for mutex and wait-notify

One of the causes of error in mutex is: faulty selection of the mutex object. The other cause of error is, differences in mutex and wait/notify handling.

For this reason the Java developer in the beginning on the 1990^th takes the wisdom about this topic and decides, that the mutex and wait/notify instances are joined together with the data. Because all data have java.lang.Object as their first base (super) class, all data have the capability to use for mutex and wait/Notify. The effort is not to high, then of course internally only a pointer (reference) is necessary, the data itself are allocated in the operation system level, only if necessary.

In Java you write for a mutual excluded access:

synchronized(this) {
  access the this.data;
}

The closing curly brace of this statement block is the unlock. If any access is written in this kind (change, read), all is mutual excluded.

If you have a wait-notify problem, you write:

//thread1
synchronized(this) {
  while(! this.bDataOk) {
    this.bWait = true;
    this.wait();
    this.bWait = false;
  }
  access the this.data;
}

//thread2:
synchronized(this) {
  prepare the this.data;
  this.bDataOk = true;
}
//
synchronized(this) {
  if(this.bWait) {
    this.notify();
  }
}

The second thread prepares the data firstly independent, but under mutex (synchronized)). Then, possible with a second synchronized maybe also later under more preparations, it checks whether anybody waits and calls then notify(). The mutex objects are all this respectively their base class Object. Of course you can work also with any other data. but be consistent.

This works proper and proven.

But because mutex and wait/notify needs effort on the operation system level, Java has introduced with the version 5 in ~ 2004 the java.util.concurrent.atomic package with the appoach described for emC in Atomic_emC.html: Lockfree programming with atomic access The atomic access is possible in many specific cases, saves calculation time and it is also possible for embedded control using in interrupts.

6. Exception, Debugging and deadlock situations

As shown in the chapter above, there are some pitfalls possible especially on unseen stupid software errors. Firstly the software should be reviewed by using style guides and rules, because, see chapter Accidence of errors on programming mistakes on simple debugging or first tests the effect may not be obvious. What helps: Long-term test under provocatively difficult conditions or in real situations.

Then a deadlock or too long waits may occur.

6.1. Independent monitor thread for log and debug support

What is helping: An independent monitor thread, which looks on all Thread_OSemC data whether the threads are waiting, and looks for all Mutex_OSemC how long they are locked, look for all WaitNotify_OSemC how long they are waiting in which threads. An abnormal situation should be logged.

If the applications blocks really, then this thread or also a manual debugging data check can help to explore the situation, and think about how to solve, where is the thinking mistake.

It is not recommended that this monitor thread programmatically resolves a deadlock. This can be only a more confusion, because the deadlock is not pre-thought.

6.2. Exception handling and ERROR_SYSTEM_emC

The Exception handling is really proven also in C language in emC, see article Stacktrace, ThreadContext and Exception handling. There is no reason to discuss whether Exception handling should be used or not.

For debugging and running under PC the C++ exception handling should be used. Also the longjmp solution works proper. If an exception occur, the stack trace (calling levels) are visible or can be locked. If the exception handling is disabled, in poor embedded environments (for fast threads the longjmp is recommended), the reaction should be tested with exception handling before.

Exceptions are elaborately thrown on programmer mistakes, but never in a proven program. The exceptions should not be part of the user algorithm, only used to ensure proper detection of programming errors.

Hence the CATCH block may be on the end of a longer functionality, not for any access. If the execution lands in a CATCH, you should think about why, what is faulty programmed.

The same is with error returns from the operation system ressources. From the view point of the operation systen there are many sources of errors, and the operation system should check all. But if all is proper programmed, operation system errors should not be occur. Hence all error returns should also THROW an exception.

The particulary used call of ERROR_SYSTEM_emC(…) for that can be used for debug firstly. You can write your own code for that, set a breakpoint there, and TROW an exception inside this routine.

6.3. What to do on a blocking application

If a system blocks in current usage, sometimes only a termination of the application may help, or termination of the whole process inside a controller, not only end individual threads.

If your application operates in a sensitive environment, such as in a rocket to a planet, a nuclear power plant, or even on an airplane, you should have a fallback device to be able to shut down and restart the faulty device, just to be able to intervene appropriately.

7. Example in testThreadMutexNotify_emC.c

The example is exact a part of module test in emC. It gives an usage pattern.

The mutex and wait/notify is done with wrapper around the OS-access operations. The "multiplatfor" programming in C/++ language should not be done regarding a specific operation system, just should be independent. The next chapter shows the adaption of the operations to the OS level.

All is contained in the source Test_emC/src/test/cpp/emC_Test_Threads/Test_ThreadMutexNotify_emC.c

First a data struct with some data and OS handle is defined:

Cpp: Test_ThreadMutexNotify_emC.c, data for test

typedef struct DataTest_ThreadMutexNotify_emC_T {

  /**Store the handle for access to the threads, not necessary here. */
  Thread_OSemC* hThread[3];

  /**Mutex instance for the data itself. */
  Mutex_OSemC_s mutexAccess;

  /**Mutex instance for the wait/notify. */
  Mutex_OSemC_s mutexNotify;

  /**Wait/notify management mechanism instances. */
  WaitNotify_OSemC_s waitNotify;

  /**State variable for synchronization. */
  bool bFirstData;//: true on first transmission of new data from v1=0
  bool bRunning;  //: set true after first data calculation
  bool bData;     //: notice that data are prepared
  bool bWait;     //: notice that the other thread is really waiting
  bool bNext;     //: notice that next data should be prepared. 

  /**Some example data which needs to be consistent
   */
  struct Data_T {
    int v1, v2;
  } data;

  /**The number of loops in the three threads. */
  int ctLoops0, ctLoops1, ctLoops2;



  /**Some counters if errors occur, remain 0 if all is ok. */
  int ctFirstData;
  int errorWaitNext;
  int errorTimeoutLockReentrant;
  int errorDiff1;
  int errorDiff2;
  int errorDataProgress;
  int error_bData;

  /**field of bits, 0x10, 0x20: run threads, 0x1, 0x2,  set on end threads. */
  int volatile bRunEnd;
} DataTest_ThreadMutexNotify_emC_s;

The data for the threads are referenced (allocated in heap), whereas the emC data for the mutex and waitNotify are stored here. It is important that this data are not copied, only one instance should be existing, but it can be referenced. That is because the inner operations should work with this only one instance. It is obvious, that allocated data may be better, it is more self-evident that they won’t be copied, only referenced. But this depends on the users programming. The [cpp} language my be more safe for that approaches because a copy constructor can be denied. In Java there are also not prone of errors, it works with allocation and referencing.

The struct Data_T {…} contains the example data which should be consistent.

The other variable are usual for test evaluation.

The start operation initializes the data:

Cpp: Test_ThreadMutexNotify_emC.c, test preparation, initializing and start the other threads

void testThreadMutexNotify_emC ( ) {
  STACKTRC_ENTRY("testThreadMutexNotify_emC");
  TEST_TRY("testThreadMutexNotify_emC");
    #ifdef DEF_ShowTime
      TimeAbs_emC timeAbs = {0};
    #endif
    DataTest_ThreadMutexNotify_emC_s* thiz = &data;
    data.bRunEnd = 0x70;
    printf("started\n");
    //--------------------------------------------------------- Create mutex and threads
    int ok = createMutex_OSemC(&thiz->mutexAccess, "mutexAccess");       // first create mutexs
    TEST_TRUE(ok ==0, "createMutexAccess successful");
    ok = createMutex_OSemC(&thiz->mutexNotify, "mutexNotify");       // because the threads are immediately started.
    TEST_TRUE(ok ==0, "createMutexNotify successful");
    bool bok = createWaitNotifyObj_OSemC("waitNotify", &thiz->waitNotify);
    TEST_TRUE(bok, "createWaitNotify");
    //                                                       //create a threads
    data.hThread[0] = main_Thread_OSemC();
    data.hThread[1] = alloc_Thread_OSemC("Thread1", threadRoutine1_Test_ThreadMutexNtify_emC, &data, 128, 0);
    data.hThread[2] = alloc_Thread_OSemC("thread2", threadRoutine2_Test_ThreadMutexNtify_emC, &data, 128, 0);
    //
    start_Thread_OSemC(data.hThread[1]);
    start_Thread_OSemC(data.hThread[2]);
    //
    printf("threads started\n");
    TimeAbs_emC timeStart; setCurrent_TimeAbs_emC(&timeStart);

    mainThreadRoutine(&data, false);
    mainThreadRoutine(&data, true);

The mutex and wait/notify instances are created, the threads are created and starts immediately, at least in the main thread the following operation is called:

Cpp: Test_ThreadMutexNotify_emC.c, thread-main data operation start and data setting

static int mainThreadRoutine ( DataTest_ThreadMutexNotify_emC_s* thiz, bool bUseReentrantLock) {

  int ctRun = 100; //determines number of expected entries in RingBuffer which are read.
  thiz->bFirstData = true;
  int baseVal = 0;
  do {
    if(++baseVal >= 10) { baseVal = 0; }
    lockMutex_OSemC(&thiz->mutexAccess); {     // write the result of data calculation under lock
      if(bUseReentrantLock) {
        bool bOkReentrant = setDataUnderLock(thiz, baseVal);    // call a given funktion which also locks.
        if(!bOkReentrant && ctRun >3) {
          ctRun = 3;                             // shorten the number of loops dont spend too much test time.
        }
      } else {
        thiz->data.v1 = baseVal;                 // because the result should accessable only consistent
        sleep_Time_emC(1);                       // -->yield other threads only to provocate accesses during. 
        thiz->data.v2 = baseVal +3;              // only an example for data calculation.
        thiz->bRunning = true;
      }
    } unlockMutex_OSemC(&thiz->mutexAccess);                 // both values should have anytime a difference of 3

Here, in a loop, under mutex, the data are written.

The next snippet shows the notify of the waiting other thread:

Cpp: Test_ThreadMutexNotify_emC.c, thread-main data operation, notify a waiting thread

    //----------------------------------------------------  Notify the other thread that data are prepared:
    lockMutex_OSemC(&thiz->mutexNotify); {                // do all under mutex lock
      thiz->bNext = false;                               // wait for notifying bNext from the other thread.
      thiz->bData = true;                                // notice data are prepared, the other thread can access
      if(thiz->bWait) {                                  // notify only if the other thread is really waiting
        notify_OSemC(&thiz->waitNotify, &thiz->mutexNotify);
      }
    } unlockMutex_OSemC(&thiz->mutexNotify);

After notifying, the other thread should evaluate the data, and this thread should only continue, if the other thread gives the info to continue. This is here the variable bNext as atomic accessed variable. It is possible (may be recommended) to use here a wait instead the showed polling, but this example uses just polling. If the other waiting thread is higher prior, it should be finished its work during 1 ms. Hence the polling is proper with view to the whole system.

Cpp: Test_ThreadMutexNotify_emC.c, thread-main data operation, wait for next step

    int timeout = 2;
    while(!thiz->bNext && --timeout >0) {        // wait till bNext flag is set by the other thread.    
      sleep_Time_emC(1);                         // yield other threads. Wait 1 ms for new inputs in Ringbuffer.
      thiz->ctLoops0 +=1;
    }
    if(timeout ==0) {
      thiz->errorWaitNext +=1;
    }
    thiz->bFirstData = false;

If this thread does not wait, it is possible that a lesser prior thread does not receive the data. That is a typical mismatch situation which may occur. Multi threading is not simple to design. Also on using event queues one thread may produce a lot of events, and the receiver is not fast enough to process it.

And now the end of the loop and the end of this operation, only to complete the docu:

Cpp: Test_ThreadMutexNotify_emC.c, thread-main end of data operation

  } while(--ctRun >=0);
  return 0;
}

One of the other thread accesses the data, use a lock to get only consistent data:

Cpp: Test_ThreadMutexNotify_emC.c, thread1

static int threadRoutine1_Test_ThreadMutexNtify_emC(void* data) {      //Thread routine which emulates the interrupt.
  int err = 0;
  STACKTRC_ENTRY("threadRoutine1_Test_ThreadMutexNtify_emC");
  printf("threadRoutine2_Test_ThreadMutexNtify_emC\n");
  TRY {
    DataTest_ThreadMutexNotify_emC_s* thiz = C_CAST(DataTest_ThreadMutexNotify_emC_s*, data);
    while(thiz->bRunEnd & 0x10) {
      thiz->ctLoops1 +=1;
      int v1, v2;
      lockMutex_OSemC(&thiz->mutexAccess); {                // write the result of data calculation under lock
        v1 = thiz->data.v1;                           // because the result should accessable only consistent
        v2 = thiz->data.v2;                        // only an example for data calculation.
      } unlockMutex_OSemC(&thiz->mutexAccess);                 // both values should have anytime a difference of 3
      int diff = v2 - v1;
      if(thiz->bRunning && diff !=3) {
        thiz->errorDiff1 +=1;
      }
      sleepMicroSec_Time_emC(10);                          //important: yield the CPU, wait a moment.
    }
    //while(true){ sleepMicroSec_Time_emC(2000); }
    thiz->bRunEnd |= 0x1;                // flag set to notice, thread operation is finished.
  }_TRY
  CATCH(Exception, exc) {
    err = 99;
  }
  END_TRY
  STACKTRC_RETURN err;
}

The whole thread operation is obviously. It runs in a loop, accesses the data under mutex, checks the consistence, counts an error if the data are non consistent. The thread runs so long as thiz→bRunEnd has set a flag bit. On end it sets a flag bit to notify its end state. This is simple, without mutex, only set and checked ones.

The other thread routine has the wait approach: It is a little more complex, thats why shown in parts.

Cpp: Test_ThreadMutexNotify_emC.c, thread2 start

static int threadRoutine2_Test_ThreadMutexNtify_emC(void* data) {      //Thread routine which emulates the interrupt.
  int err = 0;
  STACKTRC_ENTRY("threadRoutine1_Test_ThreadMutexNtify_emC");
  printf("threadRoutine2_Test_ThreadMutexNtify_emC\n");

  TRY {
    DataTest_ThreadMutexNotify_emC_s* thiz = C_CAST(DataTest_ThreadMutexNotify_emC_s*, data);
    int v1z = 0;
    while(thiz->bRunEnd & 0x20) {
      thiz->ctLoops2 +=1;
      lockMutex_OSemC(&thiz->mutexNotify); {                  // should be wrappend with mutex
        if(!thiz->bData) {
          thiz->bWait = true;                      // wait for notify, in the wait,
          wait_OSemC(&thiz->waitNotify, &thiz->mutexNotify, 100); // notice: mutex is freed inside wait)
          thiz->bWait = false;
        }
        if(thiz->bRunning && !thiz->bData) {       // check whether wait is woken up with bData, should be
          thiz->error_bData +=1;
        }
        thiz->bData = false;                       // set false for next loop
      } unlockMutex_OSemC(&thiz->mutexNotify);

The first part shows the start of the thread operation, and wait for data. But it waits only if data are not available (flag bData is false). This is queried under mutex, because elsewhere it is possible, that not bData is recognized, then the other thread is active, sets the data, checks notify, does not notify because this thread is not waiting, and resumes work, and waits for bNext. Then this thread comes back, has already recognized ! bData, does not know that data are set in meanwhile, hence waits, but nobody notifies. Hence also the other thread waits for ever or till a error-timeout, because the other thread waits for set bNext. This is a typical dead lock because faulty thread data exchange and waiting conditions.

The mutex helps, because the state ! bData queried ones remains true, because the other thread cannot access and changed. The other thread waits to get the lock for mutex instead. All is proper.

The wait has a timeout of 100 ms, just because errors cannot be excluded. Any wait should programmed with a enough great timeout to prevent hanging for ever. But if the timeout occurs, bData is false and the error is counted. For a more comprehensive situation in practice somewhat should be done, a error (log) message or what else, for this situation.

The next part, accessing the data:

Cpp: Test_ThreadMutexNotify_emC.c, thread2, access the data

      //------------------------------------------------------- access to the data
      int v1, v2;
      bool bFirstData;
      lockMutex_OSemC(&thiz->mutexAccess); {                // write the result of data calculation under lock
        v1 = thiz->data.v1;                           // because the result should accessable only consistent
        v2 = thiz->data.v2;                        // only an example for data calculation.
        bFirstData = thiz->bFirstData;
      } unlockMutex_OSemC(&thiz->mutexAccess);                 // both values should have anytime a difference of 3
      //
      thiz->bNext = true;                          // bNext set after access, does not need mutex, it is atomic.

does not need the mutex, because the data are not changed, because of programming in the other thread. But the mutex is not a high effort and not bad.

The data are accessed and copied to local data variables for evaluation, because the data creation thread should not wait for evaluation, should work furthermore. Thats why also the bNext is set here. The bData is set also here for the next loop.

Note that often set a variable is better done only in one thread, the other thread(s) only query the value.

Now the evaluation:

Cpp: Test_ThreadMutexNotify_emC.c, thread2 check data consistence

      //-----------------------------------------------------   evaluate the data.
      if(thiz->bRunning) {
        int diff = v2 - v1;              // mutex is not necessary because the other thread waits for new calculation.
        if(diff !=3) {
          thiz->errorDiff2 +=1;
        }
        if(bFirstData) {
          thiz->ctFirstData +=1;
        }
        else if( v1 !=0 && (v1 - v1z) !=1) {
          thiz->errorDataProgress +=1;
        }
        v1z = v1;
      }
    } //while thread loop

checks also, whether the data comes one after another. The distance of v1 compared with the last value in v1z should be 1, or v1 is 0. That is the condition of the data in this example. It is tested, that all data changes are received.

At least, for completion, the end of the thread.

Cpp: Test_ThreadMutexNotify_emC.c, thread 2 end

    thiz->bRunEnd |= 0x2;                // flag set to notice, thread operation is finished.
  }_TRY
  CATCH(Exception, exc) {
    err = 99;
  }
  END_TRY
  STACKTRC_RETURN err;
}

As you see, dealing with multi threading is not simple and has some pitfalls. Often using an event system is more simple. But it depends on the requests of the application, the resources and also the time for developing (using more resources, faster CPU and a more simple software, if lesser development time is available).

On end, look on the evaluation of the test as continue of the main test operation:

Cpp: Test_ThreadMutexNotify_emC.c, thread termination

    thiz->bRunEnd = 0x0;                           // the threads should terminate, atomic write access
    thiz->bRunning = false;                        // flag do no more evaluate data 
    bok = lockMutex_OSemC(&thiz->mutexNotify); {          // one thread is waiting.
      bok = notify_OSemC(&thiz->waitNotify, &thiz->mutexNotify);      // to end this thread.
    } bok = unlockMutex_OSemC(&thiz->mutexNotify);
    int ctAbort = 100;
    while(thiz->bRunEnd !=3 && --ctAbort >0) {     // wait for termination of both threads
      //wait for ending thread.
      sleep_Time_emC(1);
    }
    CHECK_TRUE(ctAbort >0, "Other Threads are stopped");
    delete_Thread_OSemC(data.hThread[1]);
    delete_Thread_OSemC(data.hThread[2]);
    deleteMutex_OSemC(&thiz->mutexAccess);
    deleteMutex_OSemC(&thiz->mutexNotify);
    deleteWaitNotifyObj_OSemC(&thiz->waitNotify);

First the termination of the other both threads. The threads checks bits in bRunEnd to end its while loop. But that is not sufficient, because thread2 may be in the wait situation, does not check the bRunEnd flag bit. Hence it should be notified, same as on data. But the flag bRunning is set to false. This is evaluated in the thread2, so it can be distinguish between 'new Data' or not.

Then the main operation waits till both threads are finished by checking the appropriate bits in bRunEnd. But this operation should also have a timeout, with the variable ctAbort. All stuff in other modules can be erroneous.

On the real end a check is done:

Cpp: Test_ThreadMutexNotify_emC.c, the end

    //
    TimeAbs_emC timeEnd; setCurrent_TimeAbs_emC(&timeEnd);
    float timeCalc = diffMicroSec_TimeAbs_emC(&timeEnd, &timeStart) / 1000000.0f;
    TEST_TRUE(thiz->ctLoops1 >1000, "Thread1 has accessed often enough: %d", thiz->ctLoops1);
    TEST_TRUE(thiz->ctLoops2 < 500, "Thread2 get bNext in a short time 1..2 ms: %d", thiz->ctLoops2);
    TEST_TRUE(thiz->ctFirstData == 2, "two times start with first data");
    TEST_TRUE(thiz->errorWaitNext == 0, "MainThread get bNext in a time < 10 ms");
    TEST_TRUE(thiz->errorDiff1 == 0, "Thread1 data are consistent");
    TEST_TRUE(thiz->errorDiff2 == 0, "Thread2 data are consistent");
    TEST_TRUE(thiz->errorDataProgress == 0, "Thread2 data are in exact order in progress");
    TEST_TRUE(thiz->error_bData == 0, "Thread2 has always data on notify");
    TEST_TRUE(thiz->errorTimeoutLockReentrant == 0, "reentrant lock in the same thread is supported.");
    #ifdef DEF_ShowTime
      OS_HandleFile hout = os_fopenToWrite("Debug/Test_ThreadMutexNtify_emCTimes.txt", false);
      if(hout ==null) {
        hout = os_fopenToWrite("build/Test_ThreadMutexNtify_emCTimes.txt", false);
      }
      if(hout !=null) {
        os_fwrite(hout, reportTimeBuffer, sizeof(reportTimeBuffer));
        os_fclose(hout);
      }
    #endif
    //Now all threads are guaranteed finished, note that there data are in this stack (!)
    TEST_TRUE(timeCalc < 0.5f, "The test runs < 0.5 seconds");

  TEST_TRY_END;
  STACKTRC_LEAVE;
}

The source can be modified to see, missing mutexes forces errors.

8. Comparison, the same example in Java

TODO The example in the chapter above should be a little bit enhanced, for example check * inversion of priority and inheritance of priority * a provoke deadlock, what to do against. * Some more

Then both should be shown in Java as also in emC.

9. Implementation of the OS routines for Windows, Linux and simple embedded control with Interrupt and Backloop

The goal of the emC 'multiplatform' is, user sources should be independent of any operation system. This includes running of (sub)modules in an interrupt/main-loop system without RTOS. It should not be necessary to adapt the sources if they should be reused in another environment.

The the both most known operation systems, MS-Windows and the Linux/Unix group, have both their specifics. Usual the whole operation system is adapted (for example Cygwin running under Windows) to execute software which were written for the other side. Exactly this is not the goal of emC, the sources itself should be portable.

Usual, for the most approaches for portable sources, not all features are necessary from a specific operation system. This makes it a little bit more simple to focus to the real important things to provide a compatibility. But it is important that the approaches are fulfilled. It is not kosher to say "xy" cannot be used under OS "z". The approaches should be fulfilled and also well checked.

In the emC sources only the implementation for Windows and Linux is done because in the moment I don’t use another RTOS system. But the adaption to also another RTOS is possible.

Due to the approach of emC, the 'multiplatform' without platform specific changes in the sources, the mutex and wait/notify statements should be also work in simple embedded control with only Interrupt and the back loop. See also Intr_vsRTOS.html For example a mutex lock can be adapted by a short 'disable interrupt' to guarantee the data consistent which may be changed in interrupt. That is described in the 3^th subchapter here.

9.1. Implementation in Windows

Generally, the OS MS-Windows has a meanwhile long history with different approaches, starting with Windows-NT and Windows-95 as first multithreading system. Some development was done regarding Windows-XP. Some changes in Windows itself were done which promises more better support as before, but the capabilities are usual the same, with not focused view to specialties of MS-Windows itself. Currently (2022/23) all is tested under Windows-10 It runs.

9.1.1. Mutex

This file contains the definition of:

Cpp: sync_OSemc.h, definition of Mutex_OSemC

#ifndef DEF_Mutex_OSemC            //maybe specific defintions may be existing. Defined in compl_adaption.h or via applstdef_emC.h
typedef struct Mutex_OSemC_T {
  const char* name;
  /**This refers OS-internal data for MUTEX allocated on createMutex and removed on deleteMutex_OSemC.
   * The internal type for pthread usage is defined in os_internals.h,
   * included only in the implementation file (os_mutex.h)
   * because OS-specific files should not be part of the user sources.
   */
  void* osHandleMutex;

  /**null then the mutex is not locked. Else: handle of the locking thread. */
  struct Thread_OSemC_T const* lockingThread;

  /**Number of lock calls of the mutex in the SAME thread. Reentrant lock should be supported. */
  int32 ctLock;

  /**Time of the last access.*/
  int32 millisecLock;
} Mutex_OSemC_s;
#endif //DEF_Mutex_OSemC

This struct contains especially the Windows HANDLE, but here as C-standard type void*. That is, because the windows-specific headers are not included here (in the user sources). This is an important principle. OS-specific headers such as <wtypes.h>, <Winbase.h> contains a lot of specifics, which is not really standard-C/++ compatible (it contains "enhancements" as saying), which may be in confusion with user sources. The void* is compatible with the Visual Studio / MS-Windows specific definition of HANDLE as you can see in the Visual Studio specific header file <winnt.h>.

In this struct also the name, the locking thread and a ctLock are contained. This data are not necessary for working, but may be a point of interest for debugging: If you see that your system has a deadlock, you may stop in debugger or get a report from any running thread, and this running thread can evaluate the data of mutexes, look which is blocked.

The file os_sync.c for MS-Windows contains:

Cpp: os_sync.c create a Mutex for MS-Windows API

extern_C int createMutex_OSemC ( struct Mutex_OSemC_T* thiz, char const* name) {
    memset(thiz, 0, sizeof(*thiz));
    CRITICAL_SECTION* cs = C_CAST(CRITICAL_SECTION*,malloc(sizeof(CRITICAL_SECTION)));
    InitializeCriticalSection(cs);
    thiz->osHandleMutex  = cs;
    thiz->name = name;
    return 0;
}

The CRITICAL_SECTION is used as Mutex object. Another possibility may be creating a CreateMutexA(…) which returns a HANDLE and using the WaitForSingleObjectEx(hHandle, dwMilliseconds, ..). The advantage may be, it has a timeout and it works also between processes (using shared memory). The CRITICAL_SECTION works only inside one process. But this is the goal, protect threads of the same process for mutual exclusion to the data. If data between processes should be synchronized, it is a special task, should be implemented not in ordinary users sources.

The Windows-Documentation says, using the CRITICAL_SECTION is fast. But the real reason is, that only a CRITICAL_SECTION reference can be used also for the wait/notify approach as Mutex, which is freed during wait without prone of error, see next chapter.

The memory for the CRITICAL_SECTION is allocated, not part of the OS-independent defined struct Mutex_OSemC, because a) its type or size should not necessary in the OS-independent enhancement file emC/OSAL/sync_OSemc.h which is included in user sources, and b) alloc is not a problem for a Windows-OS, and the point of free is clarified. The reference to this CRITICAL_SECTION data is then stored in the void* osHandleMutex.

If you want to access debug data of the CRITICAL_SECTION itself, a breakpoint inside this operation and also lock unlock, delete is possible, where the type is available. Or just specific debug operations in a monitor thread.

Cpp: os_sync.c lock a Mutex for MS-Windows API

bool lockMutex_OSemC  ( struct Mutex_OSemC_T* thiz) {
  CRITICAL_SECTION* cs = C_CAST(CRITICAL_SECTION*,thiz->osHandleMutex);
  EnterCriticalSection(cs);            // EnterCriticalSection reenter in the same thread is supported be OS-Windows
  thiz->lockingThread = getCurrent_Thread_OSemC(); //Hint: do it after lock!
  thiz->ctLock +=1;                    // count only for debug
  thiz->millisecLock = milliTime_emC();// milliseconds relative system time.
  return true;                         // true always
}

It is very simple. The EnterCriticalSection does not cause and error (returns nothing). The storing of the lockingThread, ctLock and millisecLock are only for debugging. Similar information are also available inside the CRITICAL_SECTION data. It needs only a small part of calculation time to store it. Generally such accesses for mutex are not the fastest on operation system level. The availability of debug informations may be often more important. If you need a fast calculation time, check whether an Atomic_emC.html is possible.

Cpp: os_sync.c unlock a Mutex for MS-Windows API

bool unlockMutex_OSemC  (  struct Mutex_OSemC_T* thiz){
  CRITICAL_SECTION* cs = C_CAST(CRITICAL_SECTION*,thiz->osHandleMutex);
  if(--thiz->ctLock <=0) {             // Hint: do it before unlock
    thiz->lockingThread = null;        // it is only for debugging
    thiz->ctLock = 0;
  }
  LeaveCriticalSection(cs);
  return true;
}

Unlock is similar. The info about lockingThread is removed if the mutex is no more locked, but the millisecLock remains, possible to check for debug approaches.

Cpp: os_sync.c delete a Mutex for MS-Windows API

int deleteMutex_OSemC ( struct Mutex_OSemC_T* thiz) {
  CRITICAL_SECTION* cs = C_CAST(CRITICAL_SECTION*,thiz->osHandleMutex);
  DeleteCriticalSection(cs);
  free(cs);                        // deallocates
  thiz->osHandleMutex = null;
  return 0;
}

And at least the deletion or removing of a OS-Mutex inclusively its memory area and the reference to it. The time of the last access remains also here, and also a lockingThread, which is not null if the the mutex is removed while locking (not recommended).

9.1.2. wait/notify (wait/signal)

Now look to the implementation of wait/notify.

The OS-independent file emC/OSAL/sync_OSemc.h contains the definition of:

Cpp: sync_OSemc.h, definition of WaitNotify_OSemC_s

#ifndef DEF_WaitNotify_OSemC   // it may have a specific defintiion for specific situations, containd in compl_adaption.h
typedef struct WaitNotify_OSemC_T {
  const char* name;

  /**The waiting thread, and also the owner of the mutex:
    *null if nobody waits. elsewhere a possible queue of waiting threads.*/
  struct Thread_OSemC_T const* threadWait;

  /**If the waitObj is also called by another thread, it should use the same mutex.
   * If the same waitObj is used in another situation, another mutex is possible.
   * null if nobody uses this waitObj. */
  struct Mutex_OSemC_T* mutex;

  /**The number of recursively lock should be stored to restore. */
  int ctLockMutex;

  void* osHandleWaitNotify;  // in Linux type of WaitNotify_pthread* respectively pthread_cond_t*

} WaitNotify_OSemC_s;
#endif  // DEF_WaitNotify_OSemC

This are the organization data for a wait/notify for the emC wrapper. The elements lockingThread and ctLockMutex are only used here for debugging approaches, the number of reenter call are counted, the Windows API implementation with CRITICAL_SECTION supports reenter in a mutex by the same thread. The real type of osHandleWaitNotify is the windows-internal pointer to the CONDITION_VARIABLE.

Cpp: os_sync.c for pthread, creation of WaitNotify

bool createWaitNotifyObj_OSemC  (  char const* name, WaitNotify_OSemC_s* thiz) { 
  memset(thiz, 0, sizeof(*thiz));
  //the following. mechanisms is not available in Win2000
 
  CONDITION_VARIABLE* cv = C_CAST(CONDITION_VARIABLE*, malloc(sizeof(CONDITION_VARIABLE)));
  InitializeConditionVariable(cv);
  
  thiz->osHandleWaitNotify = cv;
  return 0;
}

First, the inner data are allocated, referenced with cv, used for the InitializeConditionVariable() operation. The reference to the inner data is then stored.

Cpp: os_sync.c for pthread, wait operation

bool wait_OSemC  (  struct WaitNotify_OSemC_T* thiz, struct Mutex_OSemC_T* mutex, uint32 milliseconds) {
 //HANDLE semaphor = (HANDLE)handle;
  struct Thread_OSemC_T const* pThread = getCurrent_Thread_OSemC();
  CONDITION_VARIABLE* cv = C_CAST(CONDITION_VARIABLE*, thiz->osHandleWaitNotify);
  CRITICAL_SECTION* cs = C_CAST(CRITICAL_SECTION*, mutex->osHandleMutex);
  /*
    if(pThread != mutex->threadOwner)
    { os_Error("notify: it is necessary to have a lockMutex_OSemC in the current thread", (int)mutex->threadOwner);
      return OS_INVALID_PARAMETER;
    }
  */
  /**note the waiting thread, because notify weaks up only if a thread waits.
   */
  if(thiz->threadWait == null)
  { thiz->threadWait = pThread;
  }
  else
  { //build a queue of waiting threads. TODO
  }
  if(milliseconds ==0) { milliseconds = INFINITE; } 
  //
  //inside Sleep, the lock is free, wait for timeout or notify
  int error = SleepConditionVariableCS(cv, cs, milliseconds);
  //after sleep, the lock is given:
  thiz->threadWait = null; 
  if(error ==0) {
    error = GetLastError();
    return false;                    // time elapsed or any other error
  } else { 
    return true;                     // return >0, it is ok
  }
  //
  //the user have to be unlock.
}

First, the pointer to the inner data CONDITION_VARIABLE* cv is casted, and also to the mutex CRITICAL_SECTION*. It is checked, whether the wait operation is called under mutex. The given mutex should not be null, and the mutex should be locked with the self thread. If not, a user error is given, designated with the THROW exception. This should be detected while debugging. Also it is detected whether the own thread has a wait reference. This should not be done, it is a programming error. But this is more an assertion, it is clarified by this programming in os_sync.c.

Because the locked mutex is free during wait, the data for locking inside the mutex are copied to the wait object, and set to "free". This is not done by another algorithm, should be done here.

The next statements with threadWait stores the current thread in the wait Object. This is not necessary for the OS implementation, but it may be very interesting for debugging. Both in the Thread Object and the Wait Object it can be seen, by another running monitor thread, whether some is waiting. Because more threads may wait for one wait Object, a queue is buiit using the Thread Object itself. This is unique, because a thread can active enter only in one wait. If it does no more wait, this references are null again. All is done under mutex, so no obfuscation can occur.

If the given timeout milliseconds is == 0, it means wait infinitely, but the OS-call needs the value INFINITE for that (which is -1).

The SleepConditionVariableCS(…) is the proper Windows-API call for the wait problem. The mutex is also handled here. Inside the SleepConditionVariableCS the thread is blocked till notify, and the mutex is free!

After return from the internal wait, the mutex is already locked by this thread. First the information about reentered lock should be resored to the mutex. Also the thread queue is clarified, whereby other threads may be still wait.

The mutex which is related to the wait Object is set to null only if no other thread is waiting. It is possible to call this wait Object with another mutex in another situation, but not during the same wait and notify situation. For wait and notify always the same mutex should be used!

The operation returns false only on error, especially also on timeout.

Cpp: os_sync.c for pthread, notify operation

bool notify_OSemC  (  struct WaitNotify_OSemC_T* thiz, struct Mutex_OSemC_T* mutex) { 
  struct Thread_OSemC_T const* pThread = getCurrent_Thread_OSemC();
  CONDITION_VARIABLE* cv = C_CAST(CONDITION_VARIABLE*, thiz->osHandleWaitNotify);
  CRITICAL_SECTION* cs = C_CAST(CRITICAL_SECTION*, mutex->osHandleMutex);
  int error = 0;
  /*
  if(pThread != mutex->threadOwner)
  { os_Error("notify: it is necessary to have a lockMutex_OSemC in the current thread", (int)mutex->threadOwner);
    error = OS_UNEXPECTED_CALL;
  }
  else
  */
  {
    boolean shouldNotify = (thiz->threadWait != null); //notify only if a thread waits, test under mutex!
    //see http://msdn.microsoft.com/en-us/library/ms685071(VS.85).aspx
    if(shouldNotify)
    { 
      WakeConditionVariable(cv);
    }
  }
  return true;  //the mutex is locked.
}

The notify is more simple than wait. The mutex is not really necessary here, also not used for pthread_cond_signal but it is checked. It is a necessary style guide that also a notify should be called under the correct mutex.

Cpp: os_sync.c WaitNotify for pthread, delete operation

bool deleteWaitNotifyObj_OSemC  (  struct WaitNotify_OSemC_T* thiz) { 
  CONDITION_VARIABLE* cv = C_CAST(CONDITION_VARIABLE*, thiz->osHandleWaitNotify);
  free(cv);

  return true;
}

Delete releases the condition object and removes the memory for it. An error is possible if somewhat is faulty, for example the deleted Wait object was not initialized. This is an user software error.

9.2. Implementation for Linux using pthread

Generally, the implementation can be improved, only the following files may be modified or replaced, also by other compiling units (not overwriting the originals, exclude the originals). The goal is, fulfill the approaches of the interface definition due to chapter Approaches of mutual exclusion access for consistent data and chapter Approaches of wait/notify (wait/signal). The here presented solution runs under Cygwin (Linux emulation in Windows) and also in Debian (version 10). It uses the pthread from POSIX.

9.2.1. Mutex

The OS-independent file emC/OSAL/sync_OSemc.h contains the definition of:

Cpp: sync_OSemc.h, definition of Mutex_OSemC

#ifndef DEF_Mutex_OSemC            //maybe specific defintions may be existing. Defined in compl_adaption.h or via applstdef_emC.h
typedef struct Mutex_OSemC_T {
  const char* name;
  /**This refers OS-internal data for MUTEX allocated on createMutex and removed on deleteMutex_OSemC.
   * The internal type for pthread usage is defined in os_internals.h,
   * included only in the implementation file (os_mutex.h)
   * because OS-specific files should not be part of the user sources.
   */
  void* osHandleMutex;

  /**null then the mutex is not locked. Else: handle of the locking thread. */
  struct Thread_OSemC_T const* lockingThread;

  /**Number of lock calls of the mutex in the SAME thread. Reentrant lock should be supported. */
  int32 ctLock;

  /**Time of the last access.*/
  int32 millisecLock;
} Mutex_OSemC_s;
#endif //DEF_Mutex_OSemC

This are the organization data for a Mutex for the emC wrapper. The elements lockingThread and ctLock are necessary here. The real type of osHandleMutex is defined in

Cpp: os_internal.h for pthread, data definition for osHandleMutex:

typedef struct {                    // for internal use
  pthread_mutexattr_t attr;         // this is not a pointer, it is a specific struct
  pthread_mutex_t m;
} MutexData_pthread;

It contains OS-specifically inner data. In the Cygwin implementation the pthread_mutex_t seems to be a pointer itself, see copy from

//in Cygwin/usr/include/sys/_pthreadtypes.h
typedef struct __pthread_mutex_t {char __dummy;} *pthread_mutex_t;

But this is only a dummy definition, because Cygwin uses really Windows-API calls to fulfill the mutex. The originally, in Linux Debian looks like:

//in /usr/include/x84_64-linux-gnu/bits/pthreadtypes.h

//
typedef union
{
  struct __pthread_mutex_s __data;
  char __size[__SIZEOF_PTHREAD_MUTEX_T];
  long int __align;
} pthread_mutex_t;

// in /usr/include/x84_64-linux-gnu/bits/struct_mutex.h
struct __pthread_mutex_s
{
  int __lock;
  unsigned int __count;
  int __owner;
#ifdef __x86_64__
  unsigned int __nusers;
// ... and many more
# define __PTHREAD_MUTEX_HAVE_PREV      0
#endif
};

Of course, these are internals of the kernel with data that need not be particularly well known. This data will be allocated, and the pointer to it is stored in the void* osHandleMutex, only internally casted to the real type MutexData_pthread*.

Now look how does the operations for a mutex works:

Cpp: os_mutex.c for pthread, creation of Mutex

int createMutex_OSemC ( Mutex_OSemC_s* thiz, char const* pName) {
  int err = 0;
  thiz->name = pName;                //assume that the name-parameter is persistent! A simple "string literal"
  //
  MutexData_pthread* h = C_CAST(MutexData_pthread*, malloc(sizeof(MutexData_pthread)));
  err = pthread_mutexattr_init (&h->attr);
  //special: It does not compile, not commonly available
  //special? pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_RECURSIVE_NP);
  //special? pthread_mutexattr_setprotocol (&attr, PTHREAD_PRIO_INHERIT);
  if(err !=0){
    ERROR_SYSTEM_emC(0, "createMutex_OSemC attribute error ", err,0);
  }
  err = pthread_mutex_init (&h->m, &h->attr);
  thiz->osHandleMutex = h;             // store the handle for access!
  if(err !=0){
    ERROR_SYSTEM_emC(0, "createMutex_OSemC error", err,0);
  }
  return err;  //usual 0
}

First, the inner data are allocated, referenced with h. It contains attr and m for the mutex data used for the pthread_…init() operations.

As you see here in the comment, some "NP = NonPortable" enhancements are checked. But because they does not work under Cygwin, and supposed only not under other systems, It is not used. The following link says also:

http://www.skrenta.com/rt/man/pthread_mutexattr_init.3.html says:

"LinuxThreads supports only one mutex attribute: the mutex kind, which is either PTHREAD_MUTEX_FAST_NP for fast'' mutexes, PTHREAD_MUTEX_RECURSIVE_NP forrecursive'' mutexes, or PTHREAD_MUTEX_ERRORCHECK_NP for ``error check- ing'' mutexes. As the NP suffix indicates, this is a non- portable extension to the POSIX standard and should not be employed in portable programs.

The mutex kind determines what happens if a thread attempts to lock a mutex it already owns with pthread_mutex_lock(3). If the mutex is of the fast'' kind, pthread_mutex_lock(3) simply suspends the calling thread forever. If the mutex is of theerror checking'' kind, pthread_mutex_lock(3) returns immediately with the error code EDEADLK. If the mutex is of the ``recursive'' kind, the call to pthread_mutex_lock(3) returns immedi- ately with a success return code. The number of times the thread owning the mutex has locked it is recorded in the mutex. The owning thread must call pthread_mutex_unlock(3) the same number of times before the mutex returns to the unlocked state."

For the same problem see also: https://stackoverflow.com/questions/2821263/lock-a-mutex-multiple-times-in-the-same-thread

Hence the reenterability to a Mutex, which is necessary and defined for emC lockMutex_OSemC() is implemented by an own but proven solution, see next:

Cpp: os_mutex.c for pthread, locking of Mutex

bool lockMutex_OSemC(Mutex_OSemC_s* thiz) {
  MutexData_pthread* hmutex = C_CAST(MutexData_pthread*, thiz->osHandleMutex);
  int error;
  Thread_OSemC* hthread = getCurrent_Thread_OSemC();
  if(thiz->lockingThread == hthread) {
    thiz->ctLock +=1;                            // reentrant lock, accept it, do not lock really
    return true;                                 // but count the number of locks.
  } else {
    error = pthread_mutex_lock(&hmutex->m);         // it was not locked by the own thread, lock or wait for lock
    //--------------------------------------------> now it is locked, or we have an error
    //                                           // write the own thread, under lock, using compareAndSwap to write it back to core RAM
    void*volatile* refThread = WR_CAST(void*volatile*,C_CAST(void const**, &thiz->lockingThread)); //Note: to execute atomic, should be volatile*
    void* givenThread = compareAndSwap_AtomicRef(refThread, null, hthread);   // expect null, then write
    if(error != 0){                              // error should be 0.
      ERROR_SYSTEM_emC(0, "unknown error in pthread_mutex_lock ", error, 0);  // Note: the thread should be noted for debugging
      return false;
    } else if(givenThread != null) {             // then also compareAndSwap has failed
      ERROR_SYSTEM_emC(0, "lockMutex_OSemC htread not 0", 0,0);
      return false;
    } else {
      thiz->ctLock = 1;                          // assert that the mutex should be free before, it is the first lock.
      return true;
    }
  }
}

The own thread address (of the Thread_OSemC* pointer) are compared firstly with the stored thread data pointer. If it is the same, and also not null, then this thread has locked the mutex already. Hence, very simple, count the reentered locks.

If it is not so, either another thread has locked the mutex, or the stored thread data pointer is null. Both should not be distinguish, the mutex should be locked. If the mutex is locked already, the execution waits in the Scheduler till unlock. If the pthread_mutex_lock operation is left, then either an error is given (not expected) or the thread has the mutex locked. Because this situation may firstly not clarified, and also the thread data pointer may not be null as expected, the current thread pointer is stored only if null is stored before. This can be done simple by a atomic access, which is supported anyway. It means after the compareAndSwap operation either the thread data pointer is stored, or it is not stored and the givenThread variable contains the thread data pointer stored before.

Now it can be tested, err should be null, and also the givenThread should be null as expected, because on first and only one successfully enter to the mutex it should be null. Both errors are not expected, it seems to be any bug in the system. Hence the "hard" ERROR_SYSTEM_emC() is called, which may stop execution, restart the controller after log or such. Calling ERROR_SYSTEM_emC() is only intended for errors, which are not user errors, only for unexpected situations. The implementation of ERROR_SYSTEM_emC() should be clarified in the application. It is also possible to store only log information, or set a breakpoint there. Normally it should not be called.

Cpp: os_mutex.c for pthread, unlocking of Mutex

bool unlockMutex_OSemC(Mutex_OSemC_s* thiz) {
  MutexData_pthread* hmutex = C_CAST(MutexData_pthread*, thiz->osHandleMutex);
  Thread_OSemC const* hthread = getCurrent_Thread_OSemC();
  //----------------------------------------------- Expect, we are under lock, all other is faulty
  if(hthread != thiz->lockingThread) {           // check if the thread is correct, do not unlock the faulty mutex
    THROW_s0n(RuntimeException, "faulty thread_un unlock ", (int)(intPTR)hthread, 0);  // not an ERROR_SYSTEM, a user error
    return false;
  }
  if(--thiz->ctLock >0) {
    return true;                                 // successfully unlock of an reentrant lock of the same thread. Locking is still active.
  } else {
    thiz->lockingThread = null;                  // now it will be really unlocked (set lockingThread = null before unlock, under mutex!)
    int error = pthread_mutex_unlock(&hmutex->m);
    if(error != 0){
      ERROR_SYSTEM_emC(0, "unknown error in pthread_mutex_unlock ", error, 0);
      return false;
    }
  }
  return true;
}

Now the counterpart, unlocking. On enter in this operation, the mutex should be locked. Elsewhere the calling of unlockMutex_OSemC() is faulty in the users software. Hence the stored thread data pointer should be the current one, this is tested. The test is a helper for software verification on run time (an assertion mechanism), not really necessary, can be written also in conditional compiling. But it may be interesting on start of development.

Now, presumed or asserted, the mutex is locked by the own thread, its counter is decremented. On reaching not 0, all is done, unlock invocations should only be counted. But if the counter is ==0, is is the last unlock call, and the mustex should be unlocked. But before unlocking (under lock!), the thread data pointer is removed. It means, if another thread now calls "lock", it has the same effect, because the lock compares only with the own pointer, which has another value. The other thread enteres the pthread_mutex_lock and waits to get the mutex.

If the thread data pointer is set ` = null` after unlocking, the situation may be occur that the another thread enters the lock, successfully, and then detects another thread data pointer, which causes an error. Hence it is important that the set ` = null` is done before unlocking, still under mutex. It is the last action.

The own thread cannot interrupt and call the mutex twice, also not on multicore processing. The assumption for that is, that any thread does use only one core for this actions. It is a problem of parallel programming implementation.

An error on pthread_mutex_unlock should not occur. It is is so, it is not a user error, it is a system error. But an error occur also, if the mutex was deleted before, hence its pthread_mutex_t is null. That may be also tested here. Of course, if the user does stupid programming, for example exchange pointers in not understandable form, all may be wrong. We are in C/++ programming level.

Cpp: os_mutex.c for pthread, delete the OS Mutex resource

int deleteMutex_OSemC(Mutex_OSemC_s* thiz) {
  MutexData_pthread* hmutex = C_CAST(MutexData_pthread*, thiz->osHandleMutex);
  free(hmutex);
  thiz->osHandleMutex = null;                     // set it to null before destroy and:
  if(thiz->lockingThread !=null) {
    THROW_s0n(RuntimeException, "mutex is locked while deleting ", (int)(intPTR)thiz->lockingThread, (int)(intPTR)hmutex);  // not an ERROR_SYSTEM, a user error
  }
  int err = pthread_mutex_destroy(&hmutex->m);
  return err;
}

Should be called at last: Remove the Mutex after all is finished.

First the pointer to the OS-mutex itself is set t0 null, before removing. If shortly before this action another process has locked the mutex, it has successfully the mutex-poiner, but it also writes its thread data pointer.

Second, the thread data pointer is checked. If it is not null, the mutex is locked. This can be a user programming error (delete mutex under lock is faulty), or an unexpected situation of multiThreading (it is assumed that the mutex is never used, but it is used before.

If the thread data pointer is null, after the mutex pointer was set to null, nobody can access the mutex, it is really free. Because an interruption of another thread after the access to lockingThread will find a null-Pointer for the mutex and cannot lock (creates an error).

Hence it is clarified that the mutex is not locked on call of pthread_mutex_destroy, and an access to the MUTEX_OSemC data after destroying offers always a null-Pointer for the mutex, never a dangling reference or such.

All this hints for the implementation should assure that the programming is perfect proven. This is necessary for such operations. If their is a thinking error, it should be clarified and fixed.

9.2.2. wait/notify (wait/signal)

Now look to the implementation of wait/notify.

The OS-independent file emC/OSAL/sync_OSemc.h contains the definition of:

Cpp: sync_OSemc.h, definition of WaitNotify_OSemC_s

#ifndef DEF_WaitNotify_OSemC   // it may have a specific defintiion for specific situations, containd in compl_adaption.h
typedef struct WaitNotify_OSemC_T {
  const char* name;

  /**The waiting thread, and also the owner of the mutex:
    *null if nobody waits. elsewhere a possible queue of waiting threads.*/
  struct Thread_OSemC_T const* threadWait;

  /**If the waitObj is also called by another thread, it should use the same mutex.
   * If the same waitObj is used in another situation, another mutex is possible.
   * null if nobody uses this waitObj. */
  struct Mutex_OSemC_T* mutex;

  /**The number of recursively lock should be stored to restore. */
  int ctLockMutex;

  void* osHandleWaitNotify;  // in Linux type of WaitNotify_pthread* respectively pthread_cond_t*

} WaitNotify_OSemC_s;
#endif  // DEF_WaitNotify_OSemC

This are the organization data for a wait/notify for the emC wrapper. The elements lockingThread and ctLock are necessary here. The real type of osHandleWaitNotify is defined in

Cpp: os_internal.h for pthread, data definition for osHandleWaitNotify:

typedef struct WaitNotify_pthread_T
{
  /**see http://linux.die.net/man/3/pthread_cond_wait. */
  pthread_cond_t waitCondition;             //: the pthread handle of the waitCond
} WaitNotify_pthread;

It contains OS-specifically inner data, adequate to Mutex defined in compiler and system specific files.

Cpp: os_sync.c for pthread, creation of WaitNotify

bool createWaitNotifyObj_OSemC(char const* name, WaitNotify_OSemC_s* thiz)
{ memset(thiz, 0, sizeof(*thiz));
  WaitNotify_pthread* h = (WaitNotify_pthread*)malloc(sizeof(WaitNotify_pthread));
  //init anything?
  int error = pthread_cond_init(&h->waitCondition, null);   // use null as attribute for default behavior
  if(error !=0) {
    ERROR_SYSTEM_emC(error, "pthread_cond_init fails", error,0);
  }
  thiz->osHandleWaitNotify = h;
  return error ==0;
}

First, the inner data are allocated, referenced with h, used for the pthread_cond_init() operations. The reference to the inner data is then stored.

Cpp: os_sync.c for pthread, wait operation

bool wait_OSemC(
    WaitNotify_OSemC_s* thiz
  , struct Mutex_OSemC_T* mutex
  , uint32 milliseconds
)
{
  int error;
  WaitNotify_pthread* h = C_CAST(WaitNotify_pthread*, thiz->osHandleWaitNotify);
  //cast it from const to non-const. const is only outside!
  //the current threadcontext is nice to have for debugging - get the name of the thread.
  Thread_OSemC* threadWait = getCurrent_Thread_OSemC();
  pthread_t hThread = pthread_self();            //The mutex should be locked, check it:
  if(mutex ==null || ! pthread_equal((pthread_t)mutex->lockingThread->handleThread, hThread)){   // should be same
    //This is a user error, try to THROW an excption or ignore it.
    THROW_s0n(RuntimeException, "wait_OSemC should be called under mutex ", (int)(intPTR)hThread, (int)(intPTR)mutex->lockingThread);
  }
  if(thiz->mutex !=null && thiz->mutex != mutex) {
    THROW_s0n(RuntimeException, "more wait_OSemC with this Obj uses different mutexes ", (int)(intPTR)hThread, (int)(intPTR)mutex->lockingThread);
  }
  if(threadWait->waitObj !=null || threadWait->nextWaitingThread !=null) {
    THROW_s0n(RuntimeException, "wait_OSemC mismatch in waiting Thread ", (int)(intPTR)hThread, (int)(intPTR)threadWait);
  }
  thiz->mutex = mutex;
  thiz->ctLockMutex = mutex->ctLock;             // save for restoring
  mutex->ctLock = 0;                             // necessary because the mutex will be free inside wait
  mutex->lockingThread = null;
  //
  threadWait->nextWaitingThread = thiz->threadWait;  // maybe null if the waitObj is not used for wait till now
  thiz->threadWait = threadWait;                 // build a queue of waiting threads.
  threadWait->waitObj = thiz;
  //
  MutexData_pthread* hmutex = C_CAST(MutexData_pthread*, mutex->osHandleMutex);
  if(milliseconds >0){
    struct timespec time;
    clock_gettime(CLOCK_REALTIME, &time);
    { uint32 sec = milliseconds / 1000;
      milliseconds -= 1000*sec;  //rest is millisec
      time.tv_sec += sec;
      time.tv_nsec += 1000000*milliseconds;
      if(time.tv_nsec > 1000000000){ //overflow of nanoseconds:
        time.tv_nsec -= 1000000000;
        time.tv_sec +=1;
      }
    }

    error = pthread_cond_timedwait(&h->waitCondition, &hmutex->m, &time);
  } else { //milliseconds == 0:
    error = pthread_cond_wait(&h->waitCondition, &hmutex->m);
  }
  if (error !=0 && error != ETIMEDOUT) {         //ETIMEDOUT comes if the time has expired befor call wait, maybe possisble
    ERROR_SYSTEM_emC(99, "error in pthread_cond_wait", error, (int)(intPTR)thiz);
  }
  //----------------------------------------------- the mutex is already locked after wait:
  mutex->lockingThread = threadWait;             // restore state before wait for the mutex.
  mutex->ctLock = thiz->ctLockMutex;
  thiz->ctLockMutex = 0;
  thiz->threadWait = threadWait->nextWaitingThread;
  threadWait->nextWaitingThread = null;
  threadWait->waitObj = null;
  if(thiz->threadWait == null) {                 // only if nobody else waits
    thiz->mutex = null;                          // set the mutex to null, other mutex can be used.
  }
  return error ==0;                              // false if timeout or any other error
  //the user have to be unlock the mutex.
}

First, the pointer to the inner data WaitNotify_pthread* h is casted. It is checked, whether the wait opeation is called under mutex. The given mutex should not be null, and the mutex should be locked with the self thread. If not, a user error is given, designated with the THROW exception. This should be detected while debugging. Also it is detected whether the own thread has a wait reference. This should not be done, it is a programming error. But this is more an assertion, it is clarified by this programming in os_sync.c.

Because the locked mutex is free during wait, the data for locking inside the mutex are copied to the wait object, and set to "free". This is not done by another algorithm, should be done here.

If a timeout is given via milliseconds, it should be converted to an absolute time which is used by pthread_cond_timedwait.

If the time for pthread_cond_timedwait is expired already on entry, it returns with the error ETIMEDOUT. This is not handled as error here, because it is possible that the thread was delayed by another thread between calculation of the absolute time till enter to pthread_cond_timedwait. It can be seen as the same effect as the timeout itself. But the operation returns with false.

Inside the pthread_cond_timedwait or pthread_cond_wait the thread is blocked till notify, and the mutex is free!

The operation returns false only on error, especially also on timeout.

Cpp: os_sync.c for pthread, notify operation

bool notify_OSemC(WaitNotify_OSemC_s* thiz, Mutex_OSemC_s* mutex)
{ int error = 0;
  WaitNotify_pthread* h = C_CAST(WaitNotify_pthread*, thiz->osHandleWaitNotify);
  //the current threadcontext is nice to have for debugging - get the name of the thread.
  Thread_OSemC const* thread = getCurrent_Thread_OSemC();
  if(mutex == null || thread != mutex->lockingThread ) {
    THROW_s0n(RuntimeException, "notify_OSemC should be called only under mutex ", (int)(intPTR)thread, (int)(intPTR)mutex);
  }
  bool shouldNotify = (thiz->threadWait != null); //notify only if a thread waits, test under mutex!
  if(shouldNotify)
  { //the mutex is locked and will be locked still!
    if(thiz->mutex != mutex ) {
      THROW_s0n(RuntimeException, "notify_OSemC should be called with the same mutex as wait", (int)(intPTR)thiz->mutex, (int)(intPTR)mutex);
    }
    error = pthread_cond_signal(&h->waitCondition);
    if(error != 0) {
      ERROR_SYSTEM_emC(99, "error in pthread_cond_signal", error, (int)(intPTR)thiz);
    }
  }
  else
  { // do nothing, all is ok
  }
  return error ==0;  //the mutex is locked.
}

Cpp: os_sync.c WaitNotify for pthread, delete operation

bool deleteWaitNotifyObj_OSemC(WaitNotify_OSemC_s* thiz) {
  WaitNotify_pthread* h = C_CAST(WaitNotify_pthread*, thiz->osHandleWaitNotify);
  int err = pthread_cond_destroy(&h->waitCondition);
  if(err !=0) {
    THROW_s0n(RuntimeException, "problem on pthread_cond_destroy ", err, (int)(intPTR)thiz);
  }
  thiz->osHandleWaitNotify = null;
  free(h);
  return err ==0;
}