The access to any target system via reflection is done with an udp datagram via ethernet normally. It is possible to use a
serial connection instead with the same data structure.
1.1 Principle of communication
Topic:.refl.acc.udp.
Last changed: 2014-09-11
Connectionlessness, request and answer
The communication is connectionless. Any requester (it is a client) can send a datagram to the target (it is the server) for a request. The target answers only
for that request. The requester can be plugged off, another requester can be plugged on, without any side effect.
For TCP communication the datagram communication (UDP) is proper for that.
In some cases a request can prepare some data in the target. That is for getValueByHandle and getContinueData. The necessary table of handle is a capability of the target, it is not bound to the communication thread. The set of data
for continue data access is also a capability of the target, not bound to the communication thread too. It means, one requester
can setup the table of continue data, another requester can manage the continue data communication. Nevertheless the communication
itself can be admitted as stateless.
More as one request in one datagram, more as one answer datagrams possible:
One datagram can contain more as one request item. See datagram structure 1.2.2 Datagrams, request and answer. The answer datagram(s) contain(s) the answers for each of that requests in the same order like the request items.
Only one datagram is possible as request. But the target's answer can need more as one datagram.
One answer item is always located in one datagram.
It is possible that one request item causes more as one answer item, expecially for 1.4.1 getFields. For that case special commands are used to recognize that.
Simultaneity of more as one request but seemliness of requests from the same source:
The target is able to process more as one request from maybe several sources (requesters) independently. That is a property
of the connectionlessness.
But it should be regarded that the target may have less power as a PC hardware. The processing of a request can be need some
more milliseconds (for example 500 ms). Therefore one requester should only send the next request if all answers of the last
request are received - or a longer timeout is expired. The requester should not repeat its request in estimation of a damage
of communication in a short time (for example 100 ms) if the target does not have sent the answer. A damage of communication
is not probably, a lag of processing the request is more probable. A repitition of request would be worse the situation.
Adequat, if a requester need data cyclically, it should send the next request only if the last one was received.
Handling of communication faults, rules:
If a datagramm is lost, nothing will be repeated automatically.
The inspector communication is not a stream. It is not important to receive all data. In opposite, only current data are the
point of interest, not older data. Therefore older requests should not repeated.
If data should be changed, the change should be done either immediately, or never! An automatic repetition after a timeout
is not desired. For example, a command to switch on or off a breaker in an electrical network should be done in a moment of
1..2 seconds, or never. Think about: A command was sent, but the communication is faulty. Then another action is done alternatively.
And suddenly, the old command is executed unexpected and damaging.
The request should be sent again only by manual stimulation in responsibility of a human.
Special case: continous data transfer:
A continous data transfer from the target to any requester is a capability to inspect the data in time of a target in a maybe
fast time step. The continous data transfer should be requested like any other stuff. The answer of the request is a simple
acknowledge with some information. But in consequently the continue data transfer will be send to that requester for example for one
second with some more special datagrams. The repition of data transfer for the following time can be requested once the acknowledge-answer
was received for the last request. Hence the data transfer is continously. See 1.4.9 request continue data transfer.
The content of the field seqnr in the head is the same in all answer datagrams like in the request. The field answerNr contains a currently incremented number started with 0x01. The answerNr of the last datagram is marked with bit 7 , or with 0x80.
1.2.3 Head of a datagram
Topic:.refl.acc.telgdef.head.
Last changed: 2014-09-11
Any datagram starts with a head. This head is defined at least with 4 bytes:
nrofBytes: The number of bytes of this datagram inclusive the head.
cmdDatagram: The cmd of the datagramm distinguishes between a request- or answer-datagram and a datagram which contains continous data.
The cmdDatagram is also a possibility to distinguish between structure versions. For example the head can be changed in future. Then another
cmdDatagram is used. In than mind it is a version identification too.
For the yet known cmdDatagram the head is defined in the following form:
encryption: It is a key for encryption of content, (yet 0).
seqnr: A sequence number of the datagram. The answer datagram is sent with the same sequence number to associate the answer
to the request.
answerNr: The number of the answer datagram for one request datagram. The answerNr in a request datagram is 0. An answer datagram
starts with 1 for the first answer. The last answer is marked with bit 0x80. For example an answer with 1 datagram has the
answerNr = 0x81. An answer with two datagrams has 0x01 and 0x82. 128 answer datagrams counts from 1, 2, ..., 0x7f, 0x80. The
answerNr = 0x80 is the 128-th and always the last answer datagram. More datagrams are not possible.
nEntrant: It designates, which entrant has sent this datagram. It is possible to have more as one entrant on the same socket
connection (same PC). The entrant number is mirrored in the answer datagram. It is a negative number to distinguish from a
<@reflitem> which has a positive number at the same position. Note that an older form of communication starts a datagram without this
head.
Older datagrams, which may support for compatibility haven't a head. They contain only items (see next chapter). The first
2 words of a datagram A datagram can start with this head or can contain only items. Datagrams without this head are an older
form of communication. That is supported for compatibility yet. If the head is not given, the connection does not support
entrants and encryption.
1.2.4 Item in a datagram
Topic:.refl.acc.telgdef.item.
Last changed: 2014-08-03
An item in a datagram presents one request or one answer or one field of cmd get fields as one part of the answer.
An item starts with a ReflitemHead. After them some 1.2.5 Basic elements follows. Any item has a boundary of 4 byte. Therefore 1..3 fill-bytes with 0 can follow. The head of the item contains the
number of bytes of the whole item in its nrofBytes-field inclusively the fill-bytes. Therefore any item starts with an address which can be divide by 4 on byte-addressing CPUs.
Any item has an head of 8 bytes. It is possible to have any payload in the item or not. The nrofBytesItem defines the number of bytes of the item inclusively the head.
ReflitemHead::= <@SIZE=8>
<@0+2 SIZE?nrofBytesItem>
<@2+2?cmdItem=1..0x7fff>
<@4+4?orderItem>.
| head of the item | payload of the item | rest to 4-byte-alignment
+---------+---------+---------'---------+---------+----+----+----+----+---------+
|nrofBytes| cmd | orderItem | ...specific ... | 0 0 |
+---------+---------+---------'---------+---------+----+----+----+----+---------+
The head of each item has always 8 Byte.
nrofBytesItem it the number of bytes inclusively the item head.
cmdItem: This distinguishes between requests and answer types.
orderItem: It is adequate the sequence number of the datagram: The request item and the answer item are associated via this order identification.
1.2.5 Basic elements
Topic:.refl.acc.telgdef.baselem.
Last changed: 2014-08-03
The basic elements are used inside items of an datagram. Any of the elements in this chapter are a
Reflelement::= ...elements in this chapter...
1.2.5.1 valuea in answer datagrams
Topic:.refl.acc.telgdef.baselem.value.
Last changed: 2013-12-08
A value is presented in a short form. It contains one byte which describes the type, following by the byte presentation of
the value. Because a byte boundary of 4 is need, the rest of a reflitem is filled with 0-bytes.
That are representations of the primitive values. The values are the image of the bit representation of the type in the hardware.
It is written in big-endian always! The first byte is the type-identifier-byte.
The first 6 bytes can be used for a password. The Byte 7 is the type. The bytes 8..15 contains an image of the memory content.
The value is set in big endian, therefore right-aligned at byte 15.
Note that it is a definition of a transfered value. On cmd set value the type of the destination field decides to the type of value. For example a int32-value can be used to set some bits of
an bit field.
1.2.5.3 One String in an item
Topic:.refl.acc.telgdef.baselem.itemString.
For example in cmdGetValueByPath the item contains only one String for the access path. The length of the string is described by the length of the item in
the head. The superior definition defines the meaning of the string:
ReflItemxyz::= <@ReflItemHead?head> <@8+(head.nrofBytes-8):String-4align?meaningOfString> .
| head of the item | string in the item | rest
+---------+---------+---------'---------+----+----+----+----+----+----+----+----+----+----+---------+
| 0x14 | cmd | orderItem | T h e s t r i n g | 0 0 |
+---------+---------+---------'---------+----+----+----+----+----+----+----+----+----+----+---------+
The String has a formalistic length of 0x0c = 0x14 - 0x08 bytes, without head bytes. Because the last 2 character are set to 0, they are the boundary-4 fill bytes. They are not associated
to the used String. But formalistic it is associated to the meaningOfString.
1.2.5.4 accessPath
Topic:.refl.acc.telgdef.baselem.accesspath.
Last changed: 2013-12-08
The access path is a String. It describes the access to an Object from the root inside a target. It assumes that all data
are accessible with one root point. The syntax of the String is described in normal ZBNF:
The accesspath can contain array designations with [index] as an numerical value. If the Object is any container such as an LinkedList etc., the index describes the element in the
container. If it is an array, it should be clearly what it is. The access path should end with a dot.
The access path is used for example in a request get value by path or get fields to address the element or structure.
1.3 Table of all commands
Topic:.refl.acc.cmdtable.
Last changed: 2014-09-21
Note that some commands are defined in the past (since 2006) and that system runs still furthermore. That commands are respected
for compatibility.
It is an old style (2006 to 2010) datagram without head. The commands are item commands.
0xffff
datagram with items with order number, the items should not be in the same order in request and answer.
It is an old style (2010 to 2014) datagram with head, but with items with order number. Ordinary the entrant is set to that position, but it is set with the value 0xffff in any known applications.
0x200
cmdItems
A datagram with command items, the answer items are expected in the same order.
0x201
answerItems
A datagram with answer items, in the same order like the requested commands.
0x202
contData
Continue data.
Datagram item identifications for items with order:
Following chapters shows all known yet items. It is possible that the items are enhanced in future. The items are distinguished
with its ReflitemHead.cmdItem which has a range of 1..0x7fff.
+---------+---------+---------'---------+----+----+----+----+----+----+----+----+
| 0x1c | 0x10 | order | r o o t . a c c |
+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+
| e s s . p a t h . 0 0 0 |
+----+----+----+----+----+----+----+----+----+----+----+----+
The item consists of the 8 head bytes and the bytes representation of the String which describes the access path to the object.
The positive answer of getFields are some items, whereby one answer item describes one field. All of the items contains the same order number in the answer,
therefore they are able to associated to the request.
answerFieldMethod::= { <@answerOneFieldMethod> }
If not all <@answerFieldMethod> can written in one <@ReflDatagram>, the answer is split into more <@ReflDatagram> with the same seqnr but incremented answerNr. All <@answerFieldMethod> has the same orderNr mirrored from the Request. Because the answer datagrams are checked and sorted by its answerNr in the <@ReflDatagramHead> all fields are received in the correct order.
That string is described in a normal ZBNF syntax, not binary. The name and type are separated by a :. If it is a static array, the size is written as [size]. If it is a dynamic container, [?] is shown after the type identifier. If the type is a structured type, a ... is written on end.
+---------+---------+---------'---------+----+----+----+----+----+----+----+----+
| 0x18 | 0x14 | order | n a m e : T y p |
+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+
| e [ 5 ] . . . 0 |
+----+----+----+----+----+----+----+----+
This answer presents a field whereby the Type is a complex type with sub structure.
Type name[5];
A method is given in an adequate way (TODO, not used yet).
+---------+---------+---------'---------+----+----+----+----+----+----+----+----+
| 0x1c | 0x30 | order | r o o t . a c c |
+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+
| e s s . p a t h . 0 0 0 |
+----+----+----+----+----+----+----+----+----+----+----+----+
+---------+---------+---------'---------+----+----+----+----+----+----+----+----+
| 0x10 | 0x26 | order |0xec| float big endian | 0 0 0 |
+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+
A value is presented by a identification byte in range from 0xe0 to 0xff, or by a String length in range of 1..0xdf, following
by the byte image of the value. See 1.2.5.1 valuea in answer datagrams.
+---------+---------+---------'---------+----+----+----+----+----+----+----+----+
| 0x1c | 0x30 | order | r o o t . a c c |
+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+
| e s s . p a t h . 0 0 0 | password |
+---------+----+----+-------------------+-------------------+-------------------|
|password | | e4 | 0 0 0 0 | int32 big endian |
+---------+----+----+-------------------+-------------------+
The value is given as memory image with password, see [[Topic:.refl.acc.telgdef.baselem.SetValue.|$chapter]].
The positive answer of setValueByPath is the same like getValueByPath with the given value after set in the target.
On this example a int32 was sent to set, but the field is a float field. Therefore a float is returned. The value is the same
like the int32-to-float-conversion if the value is able to set. A value is not able to set only if it is stored on a constant
memory location (write protected or ROM).
+---------+---------+---------'---------+----+----+----+----+----+----+----+----+
| 0x10 | 0x26 | order |0xec| float big endian | 0 0 0 |
+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+
The negative value is <@failedPath>.
1.4.4 getAdressByPath
Topic:.refl.acc.cmds.getAddrByPath.
Last changed: 2014-08-03
This request gets the address in the target system of one field which is described by the access path. The address can be
used to set as value for another special field, for example to configure referenced data. The address can be used to get memory
information of the target system.
+---------+---------+---------'---------+----+----+----+----+----+----+----+----+
| 0x1c | 0x32 | order | r o o t . a c c |
+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+
| e s s . p a t h . 0 0 0 |
+----+----+----+----+----+----+----+----+----+----+----+----+
+---------+---------+---------'---------+----+----+----+----+----+----+----+----+
| 0x10 | 0x26 | order |0xdf| address big endian| 0 0 0 |
+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+
The negative answer is <@failedPath>.
1.4.5 registerHandle
Topic:.refl.acc.cmds.regHandle.
Last changed: 2014-08-03
This request registers a path in the target system. The registering searches the element by path in the target and stores
its address, length and type in a table of maximal 4096 entries (usual 1024). If the table is full, the oldest element is
searched and removed.
A request can be done with getValueByIndex only with the returned index. It is some times shorter and faster because less space in the datagram and less time for searching
the element.
+---------+---------+---------'---------+----+----+----+----+----+----+----+----+
| 0x1c | 0x23 | order | r o o t . a c c |
+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+
| e s s . p a t h . 0 0 0 |
+----+----+----+----+----+----+----+----+----+----+----+----+
The returned value is a index with check code. The check code is used to distinguish between the same index for an older registered
value and a new one.
The item contains any desired number of indices, maximal about 300 because the length of a datagram is limited. Usual 20..100
indices may be necessary because some data should be shown.
The answer contains all values for the indices in the same order of indices in the request. If more as one answer datagram
is need, then more as one answer item will be created. The following answer item contains the next values in order of indices
of the request. A request needs exact 4 Bytes per value, for the index.
The answer needs 2..9 Bytes per value depending on the data type or more bytes if Strings are returned.
It is possible to split the answer in more as one answer item, because the values, especially String values can contain more
data bytes. If 300 values are requested in one datagram, the answer may need for example 3000 Bytes. The answer contain the
index of the first variable of this answer item. The first answer item has the firstVarIndex == 0 of course.
The answer contain that index for the handle which is faulty. If more as one index are faulty then such answer is sent for
all that handle.
A faulty index can occure if
An non-registered value for the index is used.
The index is old, its position in the target was used newly.
Note that the space for indexed values are limited in any case. On the other hand about ...100 or ...1000 variable are used
in the same time if values are shown in a graphic or recorded in curves. If older graphics or curves are re-activated, the
probability that an index is re-used and therefore the old one is invalid is high. In this case a new registerRepeat action with the known path is necessary.
The index contains the real used index in Bits 11..0 for 4096 places. In the bits 31..12 the second after 1970 of registration
is stored. With 20 Bits there are about 1000000 seconds = about 11 days. After this time an older index is not able to distinguish
from a new one. A registration in the same second for a reused index is not likely because there should be registered 1024
in the same second if 1024 places are existing.
An application should remove a stored index if it is older as 11 days - or more practicable about 1 day.
An index uses 32 bit only because a lot of indices should be able to use with one request.
The negative answer is <@failedPath>.
1.4.7 valueData
Topic:.refl.acc.cmds.getValueData.
Last changed: 2013-12-08
The valueData is a Reflitem to get some data from the target in a special form:
+---------+---------+---------'---------+---------'---------+----+----+----------+
| 60 | nEntrant| encryption | seqnr |answ| | |
+---------+---------+---------'---------+---------'---------+----+----+----------+
| 44 | 0x7001 | order | .... data ... |
+---------+---------+---------'---------+-------------------+--------------------+
| .... some more data .... |
+---------------------------------------'-------------------+--------------------+
| .... some more data .... |
+---------------------------------------'-------------------+
The position of the values should be known in the target and the client by any other common description. It is not defined
in the inspector datagram. This form is used for determined data transmission from a target to the client.
The cmdItem should be correspond to the data definition.
1.4.8 setValueData
Topic:.refl.acc.cmds.setValueData.
Last changed: 2013-12-08
It is the possibility to set any values in a determined data structure which is known in the target and the requester by any
other common description. For example the requester can know positions of values by a configuartion or map file.
This datagram does not need an answer. The answer for the usage is given in the changed value. Note that <@ReflSetValue> has 16 bytes. Note that the address and position are target specific. To designate bits in a byte the position may count
Bits. For example it has the following structure:
pppppppp pppppppp pppppppp pppppbbb
where the last 3 bits are the bitposition in the byte. To get the byte position, execute position >>3.
1.4.9 request continue data transfer
Topic:.refl.acc.cmds.contData.
It is possible to request data from a target in a continously data transfer. For this request a target system should be gather
data in each step type (maybe a cycle of few 10 microseconds) and write to a buffer. The buffer accumulates data for that
some step times. If the step-time's data are accumulated, the buffer will be sent:
For example 20 values of 16-bit-integer are copied in a cycle of 100 microseconds. 40 bytes are need therefore. A datagram
can contain 1200 byte, it means data for 30 steps or 3 ms. The communication of ethernet maybe able to send 1 datagram in
3 ms. The requester on a PC receives that data, evaluates it and shows it as datastream, which can stopped and zoomed upto
a step wide of 100 µs, show a middle value, show maximums and minimums, calculate a Fast Fourier Transformation or such other.
It should be possible to set channels for that data on the target. The target stores addresses and length for each channel.
The continue data communication is initialized and the size of a block is set with
This request cleans all registered channels or sets it to default.
The sizeData is the number of bytes of one data block.
The identContData is an identifier in range 0..0xff to distinguish between more as one such transfers. For example one transfer can be done
in a short cycle, another request is done in a longer cycle.
nrofsteps: Determines the cycle of data gathering.
The answer contains the shortTimePerSeconds: If a short time step counts for example 1 µs, this value is 1000000.0. The short time step depends on the capabilities and
properties of the target, for example it is an hardware counter with a specific clock, or it is the counter of the steps of
the fast interrupt (sampling time of calculation).
posInData: The position counts from 2. The position 0 and 1 is used for the shortTime of each data block. If a position is used twice,
it may not recognized by the target.
Example:
+---------+---------+---------'---------+---------+----+----+-------------------+
| 0x1c | 0x1101 | order | pos=0x12|0xe6| 0 | scale= 1000000.0 |
+----+----+---------+-------------------+---------+----+----+----+----+----+----+
| p a t h . t o . d a t a | 0 0 0 0 |
+---------+---------+----+----+-------------------+-------------------+----+----+
In this example the data.path may refer a float variable. Its value is converted to int16 for data transfer, because the requested type is 0xe6 = int16,
see type idents in 1.2.5.1 valuea in answer datagrams. The value is stored on position 0x12 and 0x13 with 2 bytes.
There are some register requests. If they are ambigous the content of a data set are disturbed or a negative answer is sent.
A target can store all registered channels in a list (array in C) with the struct
typedef struct ChannelContDataTransfer_Inspc {
void* address;
union { float scaling; int bitshift; }
int position;
int nrofBytes;
int conversion;
} ChannelContDataTransfer_Inspc;
If the conversion is 0, only the bytes are copied. It should be a usual case. A conversion from float to int16 maybe appropriate
if the scaling is possible and more data elements are need. A conversion from a longer to a shorter integer format may be
appropriate for the second reason too. A scaling or bit shift may be a fast operation on the target's processor.
Such an answer item should use a whole datagram. It means the item starts on position 16 of the datagram. The first data block
starts at position 36 then. For 1200 data bytes the datagram length is 1236 byte.
abstimeSecondsStart: It is the seconds after 1970 usual like a timestamp in unix systems (UTC) or as a special convention the GPS seconds (yet
20 seconds difference to UTC).
microSecondsStart: The microseconds in the start second as value 0..999999 stored in 3 Byte for the first data block.
shortTime in a data block: The unit of short Time is defined in the initContData telegram. The absolute time of the first block is
given in the head. The absolute time of all other blocks can be calculated with the difference of shorttime to the first block
and the known shorttime-unit.
nrofBlocks: It determines the number of data. It is the same value as
nrofBlocks = (head.nrofByte - 24) / blocksize
blocksize: It should refer to the given value by initContData.
A datagram will be filled with all blocks though a new request is received already. The new request will processed if all
datagrams of the last request were sent:
Requester Target
|----requContData(1)------>|
| |
| {prepares and fills 1}
| {prepares and fills 1}
|<--answerContData(1,1)----|
| {prepares and fills 1}
| {prepares and fills 1}
|<--answerContData(1.2)----|
| |
... ...
|----requContData(2)------>|
... ...
| {prepares and fills 1}
| {prepares and fills 1}
|<--answerContData(1.128)----|
| |
| {prepares and fills 2}
| {prepares and fills 2}
|<--answerContData(2.1)----|
... ...
With that nested requenst and answer a continuous data flow is possible.
2 Syntax definition for binary data
Topic:.ZBNF_syntaxBin.
Written by Hartmut Schorrig, www.vishia.org. Since 2011-01-11, Latest edition 2014-08-03
2.1 Motivation
Topic:.ZBNF_syntaxBin.motiv.
Binary data can be described in the programming language C with struct{...} in headerfiles. This is usual. But the struct-description is only simple for fix data struct. If the assignment of data depends
on content in other data (cmd in header etc) then the description with struct-constructs needs some additional explanation, which are non-formalistic.
The ZBNF-description for binary data should support
exact formalistic description manual used,
Conversion of binary data to a XML-representation,
Read binary data into a Java-program (or another program if the parser would be available for other languages) using the formalistic
ZBNF-binary description.
The possibility of generating binary data from XML or inside a Java-program is not considered yet, but it seems able to do
too.
2.2 Notation as syntax prescript
Topic:.ZBNF_syntaxBin.syntax.
The syntax prescript should present how binary data are structured.
2.2.1 Basic notation
Topic:.ZBNF_syntaxBin.syntax..
The definition of a binary-component as sequence of bytes should be the same as normal ZBNF:
component::= ''syntax''.
The <@ is the basic expression to refer to binary data:
component::= <@ ...
The size of a part of binary data should be given as first element. If a constant size is given, such components can be translated
to a C-language struct-construct. For variable size see later.
component::= <@SIZE=16> ...
2.2.2 fields and position
Topic:.ZBNF_syntaxBin.syntax..
Any value at a given position with given number of bytes can be named. The name is the semanitic-meaning respectively the
variable name in a C-language struct.
<@8+2?name1> 2 bytes, big endian at position 8, named name1.
<@8-9?name1> 2 bytes from position 8 to inclusive position 9, big endian, named name1.
<@8+2%?name1> same, but little endian. It is possible to describe a mix of little and big-endian data, though it doesn't might suitable.
<@8+4:float?name2> after the : a C-language type can be written which describes the data. It is possible to use specific types, which should be known in
the evaluater of course. The float-format might be the standard-float-format of the most processors.
<@8-23:String0?name>: It is a String at position 8 to 23. If the String contains less characters then the rest is filled with 0. Note that the
String can contain 16 Characters in this example. A 0 on end is not need (not like C-language).
<@8-head.nrofBytes:String04?name>: It is a String which's end is 4-byte-aligned. The rest is filled with 0. The number of chars is calculated and store in
the field nrofBytes in a head element. It means the length of the String is variable. Note that the end is aligned, not the length of the String. The String
may start at an unaligned position.
<@8+4%float?name2>: float in little endian.
<@6+2[10..6]?name3>: It means the bits 10..6 at the integer representation in the bytes 6 and 7, big-endian.
<@6+2[11]?name4>: A single bit. It is the same like <@6+1[3]>.
<@+2?element>: This element is positioned after the last element. The position is a result of the previous elements.
The positions and length can be numeric expressions which uses the content of defined fields. Such expression are written
in parenthesis (expression):
<@0+2?nrofBytes> <@2-(nrofBytes-1)?array>
It defines one field at byte 0,1. That value is associated to the variable nrofBytes. That variable is used to define any other position, in the case the rest of the data.
<@-(SIZE-1):String04?text>: This element contains characters from the current position till the last byte. The end position is able to divide by 4 -
4 byte allignment. If the character sequence ends with 0, the 0-chars are removed before storing or evaluation. There are
the fill bytes to the 4-byte-alignment.
<@-SIZE-5_String04?text><@(SIZE-4)+4?numeric>: It is an example for a String with 4-byte alignment, following by an integer value on end of the structure. Note that the
SIZE should be determined on reading binary data by any other field. By writing binary data the length of the String may have
any desired length, it dertermines the SIZE which's value is stored.
<@+strxyzLength:String04?text>: This String with any desired length is 4-byte-aligned on its end. The length sets the variable strxyzLength which's value can be used to set a field like <@0+2?len=(strxyzLength/2)>.
..<@?element>: If the element has a known size, the size should not be given. The missing position means: after the previous element.
2.2.3 Definition of values or ranges for variables
Topic:.ZBNF_syntaxBin.syntax..
It is possible to define a value or a range with a constant expression. On parsing of binary data it does only match if the
value is proper:
<@8+2?variable=0x2e>: should contain a value 0x2e at position 8,9, named variable.
<@8+2?=0x2e>: same, a name of the field is not necessary.
<@8+2?cmd=0x7000 - 0x7fff>: A range is given.
2.2.4 The SIZE may variable often
Topic:.ZBNF_syntaxBin.syntax..
It is possible to use the variables which are defined in the component to define the size:
defines a binary component which contains an integer value in big endian at position 0, 2 bytes. The value of that integer
is named 'nrofWords'. The byte-size of that component is the value of nrofWords * 2.
The SIZE can be used as variable in other expressions, for example:
... <@10-(SIZE-1)?theRestOfData>.
This component is a byte-sequence with a variable size. From position 10 to its end the content is named theRestOfData.
This is a complete simple component with a variable size starting with 2-byte-integer which contains a number of character,
then a 4-byte-integer, then the rest from 6 to end contains the text. The SIZE is determined by a calculation with the content of position <@0+2...>.
2.2.6 Build a component with other components
Topic:.ZBNF_syntaxBin.syntax..
The following syntax describes a superior component, which contains some binary component:
The superiorCompn matches to given data only if the field cmd, defined in the child component has the value 0x14. The ! is the key symbol after them follows a name of a field of that component.
2.2.7 Presentation of text content with normal ZBNF
Topic:.ZBNF_syntaxBin.syntax..
A component may contain text-bytes. It is designated with !textSyntax. The text-bytes (ASCII-character) are described with the known ZBNF-syntax for textual representation.
The conversion of a XML-representation back to the binary data should start on the top-level syntax-component. It should detect
the proper component with the given semantic. The syntax isn't contained in the XML-tree, only the semantic. Therefore a back
transformation is not easy to do. Possibly a Xschema-notation may helpfull. Xschema describes the formal syntax of a XML-tree
in form of admissible arrangements. The Xschema-definition should name the component-identifier (syntax name). Then a back-transformation
may be able to do.
3 Organisation of binary data access
Topic:.J1C_ByteD.
.
3.1 principle
Topic:.J1C_ByteD.pr.
Last changed: 2014-08-17
How to read structured data if the structure depends on the content? A pointer to data is given, or an array of data.
In C it is usual to read the first content maybe with a specific struct, maybe as array-access:
//Negative-Pattern!!
void readData(void* data, int length)
{
int* idata = (int*) data;
int cmd = idata[1]; //the command is written on byte position 4..7
cmd = ntohl(cmd); //because it is big endian on network net-to-host long
if(cmd == myExpectedCmd){
struct MyStruct* sdata = (struct MyStruct*)(idata +2); //data after firts 8 headbytes
...etc.
Why is that pattern problematically:
Pointer casting and address calculation in C/C++ is an source of mistake often.
This example is simple. You should count bytes etc. if it is more complex.
You can forget net-to-host, or done twice (later in code twice), there is not any safety by compiler check.
Think about all big-endian conversions if you image the data with a C struct.
If a 2- or 4-byte-integer was gotten from a odd address respecively non able to divide by 4 - It isn't a problem by software
running on a PC, but some embedded CPUs can't access memory if the memory address boundary does not match!
If you use the same code - well tested on PC - for an embedded processor, and that processor has 16-bit-integer, nothing works.
Using the functions of ByteDataAccess may be a better way.
How ByteDataAccess works:
Img:../img/ByteDataAccessBase-getCmd.png
The universal super class ByteDataAccessBase contains a pointer to the data and indices to the actual positions. This basic class provides methods such
A derived class for an application offers methods like
getCmd()
which uses for example
int getCmd(){ return getInt32(4); }
The derived class knows the position of data, relativ to the start position. The user should invoke only getCmd() without some address or index calculations.
The method getInt32() (for example) regards big or little endian and it is able to read from any odd address position. It is a minimal effort for
machine code (read and shift bytes), programmed and tested only one time. If that were need to programm at user level, it
is possible of course, but effort for user.
The user should invoke:
MyByteDataAccess myByteDataAccess(); //instance for example in stack
myByteDataAccess.assign(data, length);
int cmd = myByteDataAccess.getCmd();
That's all for beginning in comparison to the negative pattern.
3.2 Availability of sources for ByteDataAccess
Topic:.J1C_ByteD.src.
The class ByteDataAccess was developed originally in Java. It is contained as part of the software component srcJava_vishiaBase in the package org/vishia/byteData/ByteDataAccessBase.java, LPGL-license. It needs less dependencies, only for the toString()-debug output the class org/vishia/util/StringFormatter.java is used. Furthermore org/vishia/util/Java4C.java which defines some annotations. All this depending classes are contained in the same software component. This Software-Component
is available as zip-File see www.vishia.org/indexDownload.html.
That Java-class is provided as translation to C- and C++-source with the Java2C-Translator. The result of Java2C-translation is readable like a manual written code. The benefit: Same functionality, same
features, tested code in two languages. The C/C++-Sources are part of the CRuntimeJavalike source pool, provided with the sourceforge.net/projects/java2c download, see also www.vishia.org/indexDownload.html.
The C-source depends on ObjectJc.c, StringJc.c and ExceptionJc.c contained in the same package.
For C++ usage in an independent environment a subset of a simple class ByteDataAccess is available.
3.3 Children
Topic:.J1C_ByteD.child.
Especially for datagram structures in communication often a header contains basic data. Depending on the header's data some
following data have a specific structure. The following data are designated usual as payload.
If a header was read with ByteDataAccess, the payload can be read with proper derived instances of BinaryDataAccessBase. To positioning the indices to the correct data, they are arranged as child of the parent, whereby the parent is used to read the header.
How children work:
+----'----+----'----+----'----+----'----+----'----+----'----+----'----+----'----+----
| | | c m d | child's data bytes | ...further child
+----'----+----'----+----'----+----'----+----'----+----'----+----'----+----'----+----
^ ^ ^
| | |
[idxBegin getCmd(); idxFirstChild ] |
' : instance for the head data | |
[idxBegin getChildData(); ] |
|
[idxBegin ... ]
The user should program:
MyHeadAccess headAccess; //instance for head data
MyChildAccess childAccess; //instance for a child
MyChildBAccess childBAccess; //instance for another type of child data
headAccess.assign(data, length);
int cmd = headAccess.getCmd();
if(cmd == expected) {
headAccess.addChild(childAccess);
data = childAccess.getChildData();
...
} else if(cmd == anotherExpected) {
headAccess.addChild(childBAccess();
dataB = childBAccess.getData();
childBAccess.addChild(childXY);
//...substructure of the child.
}
Children can consist of its children etc. Nesting any desired.
Adding a child to this, the idxCurrentChild is set to idxFirstChild (after head data) or after the yet current child. Adding a child to the child the parent of parent is too: Its idxEnd is set to the current end of all grandchildren and the idxCurrentChildEnd is set correctly. For this operations some methods should be implemented by the derived classes:
3.4 The data reference
Topic:.J1C_ByteD.PtrVal.
Hint for C/C++-usage:
The argument data is an Byte-rray in Java (byte[]). In C it is a array of bytes too:
int8_t data[1400];
The reference data consists of 2 elements: The int8_t*-pointer and the number of elements of that array. It is defined as PtrVal_int8 type:
Because that type PtrVal_int8 is used as argument type of methods, it is a call by value. But this struct of 2 primitive types are implemented as register operation often, by a well optimized compiler.
Like in Java, both the reference to the array and the length of the array is stored in one composite data structure. That
helps to minimize errors. A PtrVal_int8 data element can be filled with:
int8_t __data__[1400];
PtrVal_int8 data; //it is the reference to the array.
data.ptr__ = & __data__[0];
data.value__ = sizeof( __data__) / sizeof(__data__[0]);
An application may (should) use only the data which contains the pointer and the length.
3.5 The sizeHead value
Topic:.J1C_ByteD.sizeHead.
An instance of ByteDataAccessBase of a defined type has a specified length of the head data. It may be 0 if head data are not defined. The value of the sizeHead element of ByteDataAccessBase should be set on initializing and not changed. An older version of ByteDataAccess has defined a virtual method specifyLengthElementHead which should return a usually constant value. The newer form does not need virtual methods, which is more proper for simple
C applications.
3.6 Read and write access, expand
Topic:.J1C_ByteD.expand.
They are two kinds:
evaluate a given structure of data (read),
assemble a new structure of data (write).
For read, the first key operation is:
myByteDataAccess.assign(data, length);
For write, instead call:
myByteDataAccess.assign(data);
or
myByteDataAccess.assignClear(data);
The second method sets all elements in data to 0.
An addChild-operation is suitable for both kinds of operation:
add a child to read data in a special structure
add a child to write new data.
The difference of both operation is the handling of the idxEnd value:
For read operations the idxEnd is set on assign(data, length).
For read operations any child's idxEnd is set on on addChild(...) to the idxEnd of the parent. It means, any child my have that data.
for read operations the element bExpand is set to =false.
For write operations the idxEnd referes the position after the head data initially, use assing(data) to initialize a ByteDataAccessBase for writing.
For write operaitons the idxEnd will be expand on any addChild(...)-operation. It is done by the protected routine expand(). The expandation will be executed for any parent of child too.
For write operations after adding all children the parent will contain the correct idxEnd for the whole structure.
Reading:
The assignment of data supplies the length, usual the number of the received data bytes. That determines the ixEnd.
Adding a child sets the child's index, the idxEnd is transfered to the child too. There is a method
which returns the number of bytes in the spread between ixChild ... ixEnd to check whether a child is possible to add to check the data:
//code snippet:
while(sufficingBytesForNextChild(knownSizeOfChild)) //check whether there are data for this child
{ parent.addChild(theChild);
theChild.getSomething(); //and check it.
}
Writing:
The assignment of data supplies only the length of data, stored in the association data. The ixEnd is set to the end of the head of this ByteDataAccess-derived type. The data should be set to 0 before this operation is done
or assignClear(data) can be used to set the data to 0.
Adding a child expands ixEnd both from the child and all of its parent. Therefore
int nrofBytesToSend = parent.getLength();
can be used to quest the number of bytes which are stored lastly.
Img:../img/ByteDataAccessBase-getCmd.png.png)
.png)
