ch10_handbook:data_file_interpretation
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revision | Next revisionBoth sides next revision | ||
ch10_handbook:data_file_interpretation [2014/04/13 12:09] – bob | ch10_handbook:data_file_interpretation [2014/04/13 17:10] – bob | ||
---|---|---|---|
Line 275: | Line 275: | ||
**Intra-packet Time Stamp, IEEE 1588** | **Intra-packet Time Stamp, IEEE 1588** | ||
- | Computer Generated Data | + | ==== Data Formats ==== |
+ | |||
+ | [[Computer Generated Data]] | ||
+ | |||
+ | [[PCM Data]] | ||
+ | |||
+ | [[Time Data]] | ||
+ | |||
+ | [[MIL-STD-1553 Data]] | ||
+ | |||
+ | [[Analog Data]] | ||
+ | |||
+ | [[Discrete Data]] | ||
+ | |||
+ | [[Message Data]] | ||
+ | |||
+ | [[ARINC 429 Data]] | ||
+ | |||
+ | [[Video Data]] | ||
+ | |||
+ | [[Image Data]] | ||
+ | |||
+ | [[UART Data]] | ||
+ | |||
+ | [[IEEE 1394 Data]] | ||
+ | |||
+ | [[Parallel Data]] | ||
+ | |||
+ | [[Ethernet Data]] | ||
+ | |||
+ | ==== Time Interpretation ==== | ||
+ | |||
+ | | ||
+ | message time stamps. The RTC clock is 10 MHz, resulting in a clock resolution of 100 | ||
+ | nanoseconds. There is no constraint on the absolute value of the RTC; at recorder power on, it | ||
+ | could be initialized to zero or some random number. Some recorder vendors will preset the RTC | ||
+ | to a value based on absolute clock time, but for interoperability reasons it is unwise to assume | ||
+ | the RTC value will be related to absolute clock time in any meaningful fashion. | ||
+ | Absolute clock time comes into a recorder and is recorded much like any other data source. In | ||
+ | fact, there may be multiple time sources recorded. Time Data, Format 1 data packets are used to | ||
+ | record input time signals. Since Time Data, Format 1 packets contain both the absolute input | ||
+ | time value and the RTC clock value at the instant the absolute time was valid, these packets can | ||
+ | be used to relate RTC values to the input absolute time source. For example, if a time packet is | ||
+ | recorded with a RTC value of 1,000,000 and an absolute time value of 100: | ||
+ | the clock time of a subsequent data packet with an RTC value of 1,150,000 could be deduced to | ||
+ | be 100: | ||
+ | |||
+ | When multiple time channels are available, it is incumbent on the programmer or data | ||
+ | analyst to determine and select the best source of time for a particular data set. For example, | ||
+ | there may be separate time channels for time derived from IRIG B, Global Positioning System | ||
+ | (GPS), and an internal battery backed up clock. In this scenario, all of these time sources are | ||
+ | present in the data file as separate channels, each correlating the RTC to its own notion of clock | ||
+ | time. The software application may allow the user to select which source of time to use for a | ||
+ | given analysis. Alternatively, | ||
+ | which time channels are providing valid time. In general, each time source will provide a | ||
+ | slightly (or not so slightly) different clock time. It is usually most correct to select one time | ||
+ | channel only and to use this channel exclusively to correlate RTC time to absolute clock time for | ||
+ | all data packet types. | ||
+ | |||
+ | The stability of the RTC isn’t specified in Chapter 10 other than to require it to be at least | ||
+ | as good as a common crystal oscillator. A good grade crystal oscillator can provide stability on | ||
+ | the order of 10 ppm. Some vendors provide a RTC source considerably more stable than this. It | ||
+ | is tempting for an application to find a single time packet early in a recording and to use those | ||
+ | time values to subsequently derive clock time from relative time. It is better to use the clock and | ||
+ | relative time values from a time packet that occurs near the current data packet as the data file is | ||
+ | decoded since there is some drift in the RTC during a recording session. It also may be the case | ||
+ | that there is a jump in input clock time during a recording, such as when GPS locks for the first | ||
+ | time, or when an IRIG time source is reprogrammed. | ||
+ | |||
+ | Index and Event Records | ||
+ | Often times it is useful to make an in-memory version of the data file index. This allows | ||
+ | rapid access to recorded data packets based on time or the occurrence of events. A general | ||
+ | algorithm for reading all root and node index packets is as follows: | ||
+ | |||
+ | 1. If " | ||
+ | 2. Move read pointer to last packet of data file. Store file offset of this packet. | ||
+ | 3. If last packet data type does not equal 0x03 (Computer Generated Data, Format 3) | ||
+ | then index does not exist. | ||
+ | 4. Get the index count from the CSDW. | ||
+ | 5. For each root index contained in the packet, | ||
+ | * Read the Node Index offset value | ||
+ | * Move the read pointer to the Node Index offset value | ||
+ | * Read the Node Index packet | ||
+ | * Get the node index count from the CSDW | ||
+ | * For each node index contained in the packet read and store the time stamp, channel ID, data type, and data packet offset values. | ||
+ | 6. Read last root node index. If offset value is equal to current root node packet offset | ||
+ | (stored in Step 2) then done. | ||
+ | 7. Else the move read pointer to the next Root Index packet offset value | ||
+ | 8. Read the next Root Index packet. | ||
+ | 9. Go to Step 4. | ||
+ | |||
+ | ==== Data Streaming ==== | ||
+ | |||
+ | | ||
+ | ports using User Datagram Protocol (UDP)/IP and Chapter 10 UDP transfer headers. This is | ||
+ | normally done over an Ethernet port, but any network connection that supports UDP/IP can use | ||
+ | this method. The .PUBLISH command is used to control data streaming. Chapter 6 defines the | ||
+ | use of .PUBLISH and has numerous examples of its use. Data can be streamed to one or more | ||
+ | specific unicast and multicast IP addresses, or broadcast address. Different channels can be | ||
+ | addressed to different addresses. | ||
+ | |||
+ | It is common to publish different groups of data to different multicast groups. According | ||
+ | to RFC 3171, addresses 224.0.0.0 to 239.255.255.255 are designated as multicast addresses. | ||
+ | Different multicast address regions are designated for different purposes. According to RFC | ||
+ | 2365, Chapter 10 data streaming should be directed to multicast addresses in the Local Scope | ||
+ | address range 239.255.0.0 to 239.255.255.255. | ||
+ | |||
+ | IP multicast packets are delivered by using the Ethernet MAC address range | ||
+ | 01: | ||
+ | 23 bits of the 28-bit multicast IP address are mapped into the 23 bits of available Ethernet | ||
+ | address space. This means that there is ambiguity in delivering packets. If two hosts on the | ||
+ | same subnet each subscribe to a different multicast group whose address differs only in the first | ||
+ | 5 bits, Ethernet packets for both multicast groups will be delivered to both hosts, requiring the | ||
+ | network software in the hosts to discard the packets which are not required. If multiple multicast | ||
+ | addresses are used, be careful to choose multicast addresses that will result in different Ethernet | ||
+ | multicast addresses. | ||
+ | |||
+ | | ||
+ | packets to the software driver for decoding. This improves performance under high network | ||
+ | traffic loads. Ethernet controllers only have a limited number of multicast addresses they can | ||
+ | filter. 16 multicast addresses is a common hardware limit. If a workstation needs to subscribe to | ||
+ | more multicast addresses than the Ethernet hardware provides for, then all multicast traffic is | ||
+ | passed to the software driver for filtering, negating the benefit of multicast filtering in hardware. | ||
+ | The size of a UDP packet is represented by a 16 bit value in the IPv4 IP and UDP headers, but | ||
+ | some software implementation treat this as a signed value with a maximum value of 2^15 or | ||
+ | 32768. Because of this, the maximum size of a Chapter 10 streaming packet should be no more | ||
+ | than 32724 bytes. Physical networks have a Maximum Transfer Unit (MTU), which is the | ||
+ | largest data packet they can carry. If a UDP packet has a size larger than the network MTU, it | ||
+ | will be fragmented into smaller packets by the IP software driver before sending them over the | ||
+ | underlying physical network. The fragmented UDP packets are then reassembled into a larger | ||
+ | packet by the IP software driver at the receiving end. There is a performance penalty for this | ||
+ | fragmentation and reassembly. Better performance may be achieved by choosing a UDP packet | ||
+ | small enough to avoid fragmentation and reassembly. Regular Ethernet supports a maximum | ||
+ | size of 1500 bytes of data payload (IP header, UDP header, and UDP data) but some newer | ||
+ | Ethernet technologies support larger jumbo frames. | ||
+ | |||
+ | | ||
+ | header to the UDP data payload. Chapter 10 data packet(s) smaller than the 32k maximum size | ||
+ | will prepend the non-segmented UDP transfer header shown in Figure 6-54. A Chapter 10 data | ||
+ | packet larger than the 32k maximum size will need to be segmented before transmission, | ||
+ | will prepend the segmented UDP transfer header shown in Figure 6-55. IPv6 supports large data | ||
+ | packets, negating the need for segmented data packets. | ||
+ | |||
+ | < | ||
+ | struct SuUdpTransferHeaderNonseg | ||
+ | { | ||
+ | uint32_t uVersion | ||
+ | uint32_t uType : | ||
+ | uint32_t uUdpSeqNum | ||
+ | }; | ||
+ | </ | ||
+ | **UDP Transfer Header, non-segmented data.** | ||
+ | |||
+ | < | ||
+ | struct SuUdpTransferHeaderSeg | ||
+ | { | ||
+ | uint32_t uVersion | ||
+ | uint32_t uType : | ||
+ | uint32_t uUdpSeqNum | ||
+ | uint32_t uChanID | ||
+ | uint32_t uChanSeqNum : 8; // Channel sequence number | ||
+ | uint32_t uReserved | ||
+ | uint32_t uSegOffset; | ||
+ | }; | ||
+ | </ | ||
+ | **UDP Transfer Header, segmented data.** | ||
+ | |||
+ | Computer Generated Data, Format 3 (Recording Index) packets are meaningless in a | ||
+ | network data stream. It is necessary that they be transmitted so that Channel ID 0 data packets | ||
+ | will have contiguous sequence numbers for error detection. They should be ignored, though, | ||
+ | when received. | ||
ch10_handbook/data_file_interpretation.txt · Last modified: 2014/07/16 15:18 by bob