Dr. Hartmut Schorrig, www.vishia.org, 2021-10-01

1. Approach

The Inspector is a tool running on PC to access data on a running software on a target. The communication between the Inspector tool and the target is done usually via Ethernet communication, or maybe in two stages using a Inspector Target Proxy outgoing from PC via Ethernet and then to the target either also via Ethernet, serial or also with shared memory if the proxy runs on the same hardware as the target processor.

A running Java program is also a target which can be inspect. The inspector on target is able to build using Java and in an similar way also in C or C++.

2. Frame in a C/++ environment

The target which runs the inspector access itself inclusively all reflection information should have enough memory, especially for the const reflection information. For that ROM is sufficient (no RAM necessary). It means for poor processors with less memory this solution cannot be used, instead the Inspector Target Proxy should be used. But especially for applications on PC or rich platform controller the immediately access from the Inspector tool is possible.

The Inspector Target Proxy itself is also such a target application. It uses the same access approach, only additionally the target proxy functionality.

2.1. Necessary instance for Inspector Service on Target

=>source: src_Appl_emC/InspcTargetExmpl/InspcTargetExmpl.cpp[tag=InspcServiceData]
/**The inspector service module. */
Inspector_Inspc_s theInspector = {0};

This structure (C language) for the whole Inspector service is defined statically on main level. That is proper. Of course dynamically allocation is possible, but not necessary. Often basic structures of an embedded application are defined statically.

That is all which should be known about data instantiation.

Furthermore some explanations:

The structure is declared in

=>source: src_emC/emC/Inspc/J2c/Inspector_Inspc.h[tag=Inspector_Inspc_s]
typedef struct Inspector_Inspc_t
{ 
  union { ObjectJc object; } base; 
  ClassContent_Inspc_s classContent;   /*The sub module ClassContent should be accessible from outside to offer methods of it in the application itself.*/
  CmdExecuter_Inspc_s cmdExecuter;   /*The main cmd executer. There may be more as one {@link CmdConsumer_ifc} */
  Comm_Inspc_s comm;   /*The communication class. */
} Inspector_Inspc_s;

It consists of the three sub modules:

  • classContent: This contains data with associated operations to access the data itself.

  • cmdExecuter: It is the functionality for evaluation the commands from:

  • comm: The communication organization.

This three parts are dispersed to support the possibility of different solutions. But this solution, written in Java and translated to C language is stable since about 2011, with only some adaption and less bug fixes.

The details of the implementation of the Inspector Service are not described here, see the comments in that sources, also in Java as in the C language sources.

This source was translated formally from Java. In Java the adequate class looks like:

=>source: srcJava_vishiaRun/org/vishia/inspectorTarget/Inspector.java[tag=Inspector_Inspc_s, class_Inspector_Data]
public class Inspector
{
.....
  private static Inspector singleton; 
  
	/**The sub module ClassContent should be accessible from outside to offer methods of it in the application itself.
	 * There are only a few public ones.
	 */
	public final ClassContent classContent = new ClassContent();
	
	/**The main cmd executer. There may be more as one {@link CmdConsumer_ifc} */
	private final CmdExecuter cmdExecuter = new CmdExecuter(classContent);
	
	/**The communication class. @java2c=embedded Type:Comm. */
	private final Comm comm;

Some algorithm which are available as Java basic features, especially the reflection system but also some java.lang and java.io classes were written similar as in Java. This was the origin of the 'CRuntimeJavalike' sources which are the predecessor for the emC sources.

As you can see it is similar. Also the algorithm are similar. Hence the behavior is similar. The Inspector Service on target runs both for C/++ applications and for Java applications in a similar kind.

Hint: Do not confuse the here described Inspector Service on Target with the C language class Service_Inspc which is defined in src_emC/emC/Inspc/Srv/Service_Inspc. (2021-09). It is the incarnation of the Inspector Service especially as Simulink Function Block, also using the src_emC/emC/Inspc/Srv/DataNode_Inspc. and some more functionalities as Function Blocks in src_emC/emC/Inspc/FB/*. This functionality is not necessary for a static data inspecting, it is for dynamically approaches which are necessary for Simulink (build data from parameter field). To prevent confusion the Service_Inspc struct should be renamed, maybe only as part of the DataNode_Inspc.

2.2. Initialization and start of the Inspector Service on Target

=>source: src_Appl_emC/InspcTargetExmpl/InspcTargetExmpl.cpp[tag=InspcServiceInitStart]
void main(int nArgs, char** argsCmd) {
  STACKTRC_ROOT_ENTRY("InspcTargetExmpl");
  int erret = 0;
  TRY {
    init_ObjectJc(&theInspector.base.object, sizeof(theInspector), 0);
    // =========== construct the inspector on target, 
    StringJc sIpPort = s0_StringJc("UDP:0.0.0.0:60092");   // IP address: any hardware adapter or also 
    //StringJc sIpPort = s0_StringJc("UDP:127.0.0.1:60092"); // IP address: only via local host access 
    ctorO_Inspector_Inspc(&theInspector.base.object, sIpPort, _thCxt);
    //
    //maybe further initializations, also a init loop
    //
    // =========== after all is initialized: start the inspector service
    MemSegmJc rootAddr = INIZ_MemSegmJc(&maindata.base.obj, 0);
    start_Inspector_Inspc_F(&theInspector, &refl_InspcTargetExmpl, rootAddr, _thCxt);

As you see this is the start of the example on main.

  • The STACKTRACE_ROOT_ENTRY is necessary. Because the application should work with full reflection on a rich target the Exception handling and ThreadContext should be also available with full capability.

  • The usage of TRY / CATCH is not necessary but recommended. See ../Base/ThCxtExc_emC.html.

  • The IP address and the port is given as literal. It can be a parameter of the target device.

  • On start_Inspector_Inspc_F(…​) a second thread for the communication is created. Then the access is possible. All necessary initializations should be done. But in generally the access to the data can handle dynamically situations (null-pointer, changing of instances respectively associations etc.).

2.3. applstdef_emC.h, conditions for compilation

The ../Base/applstdef_emC_h.html should be used to compile both all sources for the Inspector service itself and also the user sources with the same file or settings.

Thats why the usage of libraries for the Inspector sources are not trivial and not recommended, see ../Style/NoLibs.html.

=>source: src_Appl_emC/InspcTargetExmpl/applstdef_emC.h[tag=InspcSettings]
#ifndef HGUARD_applstdef_emC_Project
#define HGUARD_applstdef_emC_Project
#define HGUARD_applstdef_emC  //is defined here.

//Projectspecific applstdef_emC.h

/**It seems to be a specifica in Visual Studio. 
 * The VS-File Microsoft Visual Studio 14.0\VC\include\yvals.h
 * contains a assert-message with is prevented with this define. 
 * What ist it, what means it? not clarified yet.
 * Note: This applstdef_emC.h is only for the visual studio project.
 */
#define _ALLOW_RTCc_IN_STL  //what is it? a specialism of Visual Studio??


/**Define the granularity of the ObjectJc base class: */
//#define DEF_ObjectSimple_emC
//#define DEF_ObjectJc_SIMPLE
//#define DEF_ObjectJc_REFLREF
#define DEF_ObjectJc_SYNCHANDLE
#define DEF_ObjectJcpp_REFLECTION
#define DEF_ObjectJc_OWNADDRESS
#define DEF_ObjectJc_LARGESIZE

//#define DEF_ObjectJc_LARGESIZE

/**Define of the offering of Reflection information: */
//#define DEF_REFLECTION_NO
//#define DEF_REFLECTION_SIMPLE
//#define DEF_REFLECTION_OFFS
#define DEF_REFLECTION_FULL


//If set then the target should not use string operations
//#define DEF_NO_StringJcCapabilities


//#define USE_BlockHeap_emC
//#define DEF_BlockHeap_GARBAGECOLLECTOR


//If set, without complex thread context, without Stacktrace
#define DEF_ThreadContext_HEAP_emC
#define DEF_ThreadContext_STACKTRC

//#define DEF_Exception_TRYCpp
#define DEF_Exception_longjmp
//#define DEF_Exception_NO


//If set, no assertion is done:
//#define ASSERT_IGNORE_emC



//
//What to start as main:
//
#undef DEF_INSPC_REMOTEACCESS



/**This is to compile C++ classes of emC if __cplusplus is set.
  For C compilation this is ineffective because __cplusplus is necessary too*/
#define USE_cplusplus_emC
#define VIRTUAL_emC virtual

#define DEFINED_getVarAddrType_CalcExpr




//including the project specific reflOffs.h defines DEF_REFLECTION_OFFS 
#ifdef DEF_REFLECTION_OFFS
  //contains DEF_REFLOFFS_...for all defined ClassJc
  #include <emC_Exmpl_Ctrl/genRefl/emc_Exmpl_Ctrl.reflOffs.h>
  //Note: the adequate *.reloffs.c should be part of the project:
#elif defined(DEF_REFLECTION_FULL)
  #define DEF_ClassJc_Vtbl    //It is used in the inspector sources
#endif


#define kMaxPathLength_FileDescription_OSAL 512
#define sizeSafetyArea_allocMemC 256




#include <compl_adaption.h>
#include <emC/Base/Assert_emC.h>

#include <emC_srcApplSpec/applConv/EnhanceRef_simple.h>
#include <emC/Base/Exception_emC.h>



//only for this test application:
extern_C void outTestConditions ( );

#endif //HGUARD_applstdef_emC_Project

2.4. Necessary sources

This chapter describes which sources from emC/ are necessary to use the Inspector Target Service. It is explained for which the sources are necessary. Some sources are commonly necessary.

See the reference Visual Studio Project in Test_emC/IDE/VS_InspcTargetExmpl/InspcTargetExmpl.vcxproj

2.4.1. Sources from emC/Base and application specific sources

src_emC/emC/Base/Assert_emC.c
src_emC/emC/Base/CheckStack_emC.c
src_emC/emC/Base/Endianess_emC.c
src_emC/emC/Base/Exception_emC.c
src_emC/emC/Base/ExceptionCpp_emC.cpp
src_emC/emC/Base/ExcThreadCxt_emC.c
src_emC/emC/Base/Formatter_emC.c
src_emC/emC/Base/MemC_emC.c
src_emC/emC/Base/Memfree_ThreadAndBlockHeap.c
src_emC/emC/Base/ObjectJcpp_emC.c
src_emC/emC/Base/ObjectRefl_emC.c
src_emC/emC/Base/ObjectSimple_emC.c
src_emC/emC/Base/os_common.c
src_emC/emC/Base/Reflection_emC.c
src_emC/emC/Base/ReflectionBaseTypes_emC.c
src_emC/emC/Base/Simple_emC.c
src_emC/emC/Base/String_emC.c
src_emC/emC/Base/StringBase_emC.c
src_emC/emC/Base/StringBuilder_emC.c
src_emC/emC/Base/Time_emC.c
src_emC/emC/Base/Timeconversions_emC.c
src_emC/emC_srcApplSpec/applConv/ExceptionPrintStacktrace_emC.c
src_emC/emC_srcApplSpec/applConv/ObjectJc_allocStartup_emC.c
src_emC/emC_srcApplSpec/applConv/UmlContainer_Dummy.c

This files are necessary and sufficient for any application which uses the same full capability settings in applstdef_emC.h and which uses also String processing. This sources are not specific only for the Inspector on Target usage.

See Test_emC/IDE/VS_StringFileExmpl/StringFileExmpl.vcxproj as example.

2.4.2. Core sources of Inspector Service on Target

src_emC/emC/Inspc/Srv/CheckPwd_MinDefault_Inspc.c
src_emC/emC/Inspc/J2c/AnswerComm_ifc_Inspc.c
src_emC/emC/Inspc/J2c/ClassContent_Inspc.c
src_emC/emC/Inspc/J2c/CmdConsumer_ifc_Inspc.c
src_emC/emC/Inspc/J2c/CmdExecuter_Inspc.c
src_emC/emC/Inspc/J2c/Comm_Inspc.c
src_emC/emC/Inspc/J2c/InspcDataExchangeAccess_Inspc.c
src_emC/emC/Inspc/J2c/InspcDataInfo_Inspc.c
src_emC/emC/Inspc/J2c/Inspector_Inspc.c
src_emC/emC/Inspc/J2c/SearchElement_Inspc.c

This sources except CheckPwd_MinDefault_Inspc.c are also available as Java sources in srcJava_vishiaRun/org/vishia/inspectorTarget, see ../../../Java/docuSrcJava_vishiaRun/org/vishia/inspectorTarget/package-summary.html

2.4.3. Sources for Socket communication

The Inspector on Target Sources uses an "Interprocess Communication" approach which abstracts from usage sockets. This is a may unnecessary frame but it runs. In Java it is the srcJava_vishiaRun/org/vishia/inspectorTarget, a really pure interface.

For the C language implementation this interface is defined in

src_emC/emC/Ipc/InterProcessComm.h

their as

typedef struct InterProcessComm_t
{ union{ ObjectJc object; } base;
}InterProcessComm_s;

It uses the dynamic linked operation call in C language (not virtual with C++. There are two reasons for that decision:

  • The sources should be run in the year 2006..2011 also in C language on an embedded target. Meanwhile this is no more in focus, because usual all target compiler can be offer C++.

  • The system of virtual operations in familiar C++ environments are potentially unsafe.

The second reason is furthermore true and the implementation of virtual operations in C language runs well.

For usage of the Inspector Service on Target you need not know this concept. You should only use the implementation sources:

src_emC/emC/Ipc/InterProcessCommSocket.c
src_emC/emC/Ipc/Ipc2c/InterProcessCommFactorySocket_Ipc.c
src_emC/emC/Ipc/InterProcessComm.c

Furthermore you need the Operation System Adaption (OSAL) for sockets, for MS-Windows this is:

src_emC/emC_srcOSALspec/osal_Windows/os_socket.c

This sources should be linked for Windows against the

ws2_32.lib

For other systems the OSAL should be adapted, see ../OSHAL/OSAL.html.

2.4.4. Jc and J2c sources

This sources were immediately translated from Java to C language (J2c/) or written in C language adequate to the Java functionality (Jc/). They are:

src_emC/emC/J1c/ByteDataAccessBaseJc.c
src_emC/emC/J1c/StringFunctionsJc.c
src_emC/emC/J1c/StringFunctions_CJc.c
src_emC/emC/J1c/StringPartJc.c
src_emC/emC/J1c/StringPartScanJc.c
src_emC/emC/Jc/ArraysJc.c
src_emC/emC/Jc/CharsetJc.c
src_emC/emC/Jc/FileIoJc.c
src_emC/emC/Jc/LocaleJc.c
src_emC/emC/Jc/MathJc.c
src_emC/emC/Jc/ObjectJc.c
src_emC/emC/Jc/OsWrapperJc.c
src_emC/emC/Jc/PrintStreamJc.c
src_emC/emC/Jc/ReflectionBaseTypesJc.c
src_emC/emC/Jc/ReflectionJc.c
src_emC/emC/Jc/ReflMemAccessJc.c
src_emC/emC/Jc/StringBufferJc.c
src_emC/emC/Jc/StringJc.c
src_emC/emC/Jc/SystemJc.c
src_emC/emC/Jc/SystemOutJc.c
src_emC/emC/Jc/ThreadJc.c

An interesting source may be ByteDataAccessBaseJc.*. In Java this is

It implements a possibility to access the byte given data of a (socket) datagram (payload) with the conversion from/to Java data, usual numbers and Strings. In C/++ language such an conversion may be done often with pointer casting: The proper pointer to the part of the telegram is casted to the C language type. The next fact is the network endianness and the host endianness, whereby typically such conversion functions ntoh etc ("net to host") are called. This is not possible in Java. But the access to the bytes of the telegram with the type byte[] is possible, and a simple shift’n add or mask to build the Java data. The endianness problem is solved by the way, accessing and shifting the correct bytes. This solution is also used in C/++ language with the same algorithm. The advantage is: No confusion on casting and no confusion on maybe twice swapped ntoh etc. The calculation time for this access is less, because any processor and also Java can fast handle with shift, mask and add operations. Alternatives in Java in the packages java.nio are not faster.

The other interesting sources are the StringPartJc.c and StringPartScanJc.c with its counterparts

This sources were the oldest ones, firstly written in C++ language in the 1990th in a time where C++ has not sufficient support for String handling, then re-written for Java (2003/04) and then also translated to C language.

This classes are also the base for some parsing solutions also in C++ as in Java.

2.4.5. Operation Sytem adaption (OSAL)

That are the files for implementation on Visual Studio under MS-Windows:

src_emC/emC/OSAL/Environment_OSALemC.c
src_emC/emC_srcOSALspec/osal_Windows_Msc15/os_atomic.c
src_emC/emC_srcOSALspec/osal_Windows/os_environment.c
src_emC/emC_srcOSALspec/osal_Windows/os_file.c
src_emC/emC_srcOSALspec/osal_Windows/os_mem.c
src_emC/emC_srcOSALspec/osal_Windows/os_mutex.c
src_emC/emC_srcOSALspec/osal_Windows/os_sync.c
src_emC/emC_srcOSALspec/osal_Windows/os_thread.c
src_emC/emC_srcOSALspec/osal_Windows/os_time.c

The os_socket is mentioned also in the chapter above. This sources may be necessary too for other applications using threads and mutex. They need to be adapted on a operation environment if they are not existing. The header files which describes the interface are contained all together in

src_emC/emC/OSAL/

This header files should be implemented for a given platform.

See also link:../OSHAL/OSAL.html

3. Role of reflection

This role is essential. The reflections contain the structure information of all data.

The reflection data and its generation is described in ../Base/ClassJc_en.html

The reflection are usual stored in a refl/ sub directory beside the header. It looks like (shortened)

const struct Reflection_Fields_InspcTargetExmpl_s_t
{ ObjectArrayJc head;
  FieldJc data[3];
} refl_Fields_InspcTargetExmpl_s =
{ INIZ_ObjectArrayJc(refl_Fields_InspcTargetExmpl_s, 3, FieldJc, refl_FieldJc, ID_refl_FieldJc)
, {
    { "bRun"
    , 0   //no Array, no Bitfield
    , REFLECTION_int32
    , (4<<kBitPrimitiv_Modifier_reflectJc) //bitModifiers
    , (int32)(OFFSET_IN_STRUCT(InspcTargetExmpl_s, bRun) - OFFSET_IN_STRUCT(InspcTargetExmpl_s, base.obj))
    , 0  //offsetToObjectifcBase
    , &refl_InspcTargetExmpl
    }
  ,

....

#define DEFINED_refl_InspcTargetExmpl
const ClassJc   refl_InspcTargetExmpl =
{ INIZ_objReflId_ObjectJc(refl_InspcTargetExmpl, refl_ClassJc, ID_refl_ClassJc)
, "InspcTargetExmpl"
, 0
, sizeof(InspcTargetExmpl_s)
, (FieldJcArray const*)&refl_Fields_InspcTargetExmpl_s  //attributes and associations
, null  //method
, &superClasses_InspcTargetExmpl_s.head.object  //superclass
, null  //interfaces
, 0
#ifdef DEF_ClassJc_Vtbl
, null  //virtual table
#endif
};

They are const data.

To compile it it should be included in the appropriate C/++ files to the header, which defines the data structs.

#ifdef DEF_REFLECTION_FULL
  #include <genRefl/InspcTargetExmpl.crefl>
#elif !defined(DEF_REFLECTION_NO) && !defined(DEFINED_refl_InspcTargetExmpl)
  //Not defined with DEF_REFLECTION_OFFS but necessary, only as type marker:
  ClassJc const refl_InspcTargetExmpl = INIZ_ClassJc(refl_InspcTargetExmpl, "InspcTargetExmpl");
#endif

This sequence is described in ../Base/ClassJc_en.html#inclRefl

The reference to the ClassJc const* reflection is part of a ObjectJc base structure possible for any instance. But not all instances need be marked so. More exactly, it is possible that no instances are marked with their reflection reference. The only one root reflection reference is given on

  start_Inspector_Inspc_F(&theInspector, &refl_InspcTargetExmpl, rootAddr, _thCxt);

All reflection for referenced instances from the root may be come from the reflection of the parent, firstly from this root. Then referencing and reflection are not synchronized parallel. They are only synchronized because the reflection and the sources are compiled together with same settings. But this system works and it is proven in practice. A mistake can only occure on build errors, combining old and newer compiling results because the make time stamp or newly check was faulty or non-rebuild old libraries are used. This situation is also known on other error situations.

Because the reflection contains calculated positions on data on compile time errors cannot be occured if the sources are not maintained exactly. In such cases new elements may be missing. But the positions are correct. Removed (or renamed) elements causes a compiler error. Then latest the reflection generation should be repeated.

For derived data: The reference is type of a basic type, without own reflection in the referred instance only the basic data (of the super type) can be seen. For that the own ClassJc const* reflection as part of ObjectJc is helpfully to get the instance of a derived type.

If a type contains an ObjectJc base structure, the reflection is always gotten from there. It means, dynamically linkage and changed references (association, not aggregation in UML slang) are supported.

4. How does it work

This chapter should describe the principles, maybe to support setting a breakpoint to study the behavior or to search a sophisticated possible error on usage.

4.1. The communication thread

After

start_Inspector_Inspc_F(...);

in the user’s startup code a thread is created. The tread start routine is

//Source: emC/Inspc/J2C/Comm_Inspc.c
void run_Comm_Inspc_F(ObjectJc* ithis, ThCxt* _thCxt)
{ ....

You can set there a breakpoint to check what the thread does.

Firstly

openComm_Comm_Inspc(thiz, true, _thCxt);

is called. It opens also the InterprocessComm, the Socket communication. This can fail, for example if the port is in use already, or for any other socket reason. Then this thread is removed, with some error hints.

If the open is successful, in a while loop

receiveAndExecute_Comm_Inspc(thiz, _thCxt);

is executed. In this routine the socket receive is called. It waits without timeout for an incomming telegram from the inspector tool. You can set a breakpoint to see what’s happen after the receiveData(…​)

//Source: emC/Inspc/J2C/Comm_Inspc.c
    ipcVtbl.mtbl->receiveData(....);
    if(thiz->state != 0xd)

If receiving is ok,

cmdExecuterVtbl.mtbl->executeCmd(...)

is called. This is a virtual operation in C language. It means static code analyze is not proper. Test in debug run time. The routine is:

//Source: emC/Inspc/J2C/CmdExecuter_Inspc.c
/**Executes the given command received with this datagram*/
bool executeCmd_CmdExecuter_Inspc_F(CmdExecuter_Inspc_s* thiz, int8ARRAY buffer, ....
{ Vtbl_CmdExecuter_Inspc const* mtthis = ....

This routine analyzes the payload of the received telegram. It can contain more as one datagram for an inspector command, but usual only one. For the datagram

  cmdConsumerVtbl.mtbl->executeMonitorCmd(...)

is called. This lands in

//Source: emC/Inspc/J2C/ClassContent_Inspc.c
int32 executeMonitorCmd_XXXXi_ClassContent_Inspc(ObjectJc* ithis, struct Inspcitem_InspcDataExchangeAccess_Inspc_t* cmd, struct InspcDatagram_InspcDataExchangeAccess_Inspc_t* answer, int32 maxNrofAnswerBytes, int32 accessLevels, MemC telg, ThCxt* _thCxt)
{ ClassContent_Inspc_s* thiz = (ClassContent_Inspc_s*)ithis;

The naming with some XXX comes from the Java2C translating and has no special meaning. This routine analyzes this one inspector command, calls the proper operation and fills the return data in the answer reference.

5. Further TODO

TODO missing access via index, is yet still not implemented, and some other details