One problem faced by anyone who wants to write a data collection program is what type of real-world interface to use. There is no standard method by which the hundreds of available analog to digital converters may communicate with a PC program. When writing Radio-SkyPipe I decided to use parallel port driven ADCs that were simple and inexpensive to build and yet provided good performance. The MAX187 (single channel) and MAX186 (eight channel) 12 bit ADCs were chosen. In addition, I added sound card sampling capability for ultimate simplicity when data was being collected from an audio source such as a radio.
Many users have asked if Radio-SkyPipe could be re-designed to support the particular type of ADC that they happen to own. As some of these units cost many hundreds of dollars, one can understand why they don't want to discard them and start anew. There are serious programming problems that arise from trying to support a vast array of devices. The overhead of maintaining this software would be enormous, and it is impossible for me to purchase and test each of the devices. Still, I wanted to do something to make SkyPipe a more universal solution, so I developed a User Data Source model to help bridge the ADC gap.
The UDS model is an open source code program that sends data to Radio-SkyPipe via a TCP data link. The program could be written in any language as long as it adheres to the data exchange structures expected by Radio-SkyPipe. For now, I am providing a Visual Basic skeleton program framework upon which the user can build their UDS, but I hope to also offer skeleton programs in Delphi, Java, and C in the future. It is possible that skeleton programs in other languages may also be donated by users.
The UDS program does the actual interfacing with the data collecting device. It then formats this data and sends it along to Radio-SkyPipe with or without timestamps via a TCP connection. By using TCP, the UDS can exist either on the same computer as Radio-SkyPipe or even on a different computer on the other side of the world. SkyPipe does not care where the UDS resides as long as it can connect to it. Because UDS programs can be written in non-Windows specific languages, it becomes possible to have the UDS in a Linux, MAC, or even a non-PC device with embedded networking!
As you may have gathered, the UDS concept relies on the ability of the end user to produce program code and to compile it into an executable program. This is not possible for everyone. However, it is hoped that those who do compile UDS executables will share these with others via the Radio-Sky website, and that a pool of available UDS interfaces for a variety of ADCs will be generated. The Visual Basic UDS skeleton that is provided is heavily commented and modularized to the point that it should be very easy for the relatively inexperienced programmer to add what code might be needed to run their particular device. Many device manufacturers such as Pico and Dataq provide DLLs and example source code that can lead to very easily implementing the data retrieving routine into the VB UDS structure. The effort then becomes mostly cut and paste of the code and compiling of the executable.
Timing and Timestamps
The ability of the UDS to generate its own timestamps helps overcome a terrible problem with Windows data collection programming. Microsoft Windows platforms share CPU time among many applications. Even if you think you only have a single application running, you really do not. There are often numerous applications running in the background and the operating system itself uses CPU time. As a result, the data collection program receives attention from the computer in a sporadic and unpredictable way. Data cannot be collected evenly over time unless some external device is timing the sampling and buffering the data. By passing the timestamp generation ability to the UDS, it can in turn base these timestamps upon an external device
that is not at the mercy of Windows for time slots. Presently, Radio-SkyPipe generates a timestamp for each sample (or array of samples in a multiple channel file).
Push or Poll
A UDS may be configured to "push" data to SkyPipe. In this mode, the UDS decides when it wants to send the data to be charted. The push mode is best used when the data sampling timing needs to be controlled by the UDS or the hardware. In push mode, SkyPipe waits passively for data to be sent by the UDS. When data is received it is strip charted. The other option is to have SkyPipe "poll" for data from the UDS. When using polling, SkyPipe relies on it's software timers to trigger a request to the UDS for more data. The request is sent to the UDS and the UDS provides the data in response to poll. Note that whether the data is poll or push the option still exists to have the data timestamped by SkyPipe or the UDS.
Communications
As of Radio-SkyPipe version 2.5.0, there are two communication paths that may be used between RSP and a UDS. Originally there was only a TCP network channel. This is especially useful for cases where the UDS does not reside on the same computer with RSP. The new possible path is a direct serial connection. This was developed to support Arduino and similar microcontroller based UDSes where the intelligence normally supplied by a separate UDS executable may be contained within the microcontroller code. In other words, the data gathering device needs no interpreter software residing on the PC. The device itself communicates using the UDS protocol..
Sockets
The UDS also communicates with SkyPipe using Windows "sockets". A socket is a software interface with the networking hardware that allows you to perform the necessary functions to communicate using the TCP network protocol. Sockets hide most of the technical details of the communication process from you. In the VB example and in Radio-SkyPipe itself I use an ActiveX control called the SocketWrench control to make this even easier. The SocketWrench control must be added to your Tools palette when using the VB skeleton. SocketWrench is free and
may be downloaded from http://www.catalyst.com/
. SocketWrench is a fancy but easy to use interface to the Microsoft Winsock component and thus the Microsoft Winsock must also be present on your computer. Of
course, if a UDS is created in another language you may not want to or have the
ability to use the SocketWrench control. You should be able to perform the same
functions with other Winsock type components.
A UDS is a "server" in the strictest sense of the word. When the UDS starts, a socket is created that "listens" for a connection request from a "client" over the network. SkyPipe acts as the client and sends this request to connect to the UDS. When the request is received, the UDS socket Accepts the connection and creates a new socket that actually handles the connection. Data is then passed via this new socket. It would be possible with some modification of the code to have a UDS accept multiple SkyPipe connections from different PCs, but the goal of the UDS skeleton is to remain simple and understandable and so only one connection is provided.
The Initial Exchange
After the connection to SkyPipe has been established, a few bits of information are exchanged between the UDS and SkyPipe before the sample data begins to flow.
1. SkyPipe sends an INIT command to the UDS. When the UDS receives the INIT command it calls a procedure in its own code where anything that needs to be done by the UDS software prior to data collection can be accomplished. This might be a routine that initializes the ADC, for example, or perhaps just the initialization of certain variables by the UDS.
2. When the UDS has finished its initialization tasks, it sends back a code to SkyPipe that informs it of its success.
3. SkyPipe then sends a POLL or PUSH command to the UDS informing it of which mode of operation that SkyPipe is expecting to use. Polling or pushing of data is determined in the UDS configuration screen within SkyPipe. When the UDS receives the POLL or PUSH command it calls the appropriate routine (POLLinit or PUSHinit) in its own code to set up this mode of operation if anything special needs to be done.
4. SkyPipe then sends the get channels command ( GETC ) to the UDS. The UDS responds with the number of data channels that it will be using.
5. Finally, SkyPipe sends a STAT command to the UDS. This is the last chance for the UDS to do what it needs to do before beginning the actual streaming of sample data to SkyPipe. A code is sent back to SkyPipe that everything is ready to go and the actual data collection process can begin. If a timer needed to be turned on by the UDS, this would be a good time for this to be done. If the UDS is going to generate its own timestamps then it must tell SkyPipe approximately where the chart will start in time and so it sends SkyPipe a timestamp using a special command. SkyPipe then redraws the empty
strip chart beginning at this point in time.
Sample Data Exchange
POLL Exchange
1. SkyPipe sends a GETD command to the UDS.
2. The UDS receives the GETD command and calls the data collection routine GetSendData in its own code. In turn GetSendData calls any subroutine that gathers data from the actual device.
3. Data from each channel is then sent back to SkyPipe the appropriate channel number. If there is only one channel then the channel number is 0, otherwise it is 0 - 7. The channel data is sent as a "#" plus a one character string of the channel number, followed by a string representing the actual sample value (such as "2387.4 , or -137), and finally ending with a single byte ASCII character 255.
4. After sending data for each available channel, the UDS then sends one of two commands. If the UDS is generating its own timestamps it sends SkyPipe a command number string "^^3000" plus a string representing the timestamp in VB date format, plus a single byte ASCII character 255. Note that all data exchanges end with this character. If the UDS wants SkyPipe to generate the timestamp then it just sends the command "^^3001" plus a single byte ASCII character 255.
PUSH Exchange
Data is sent to SkyPipe in exactly same fashion as for POLL data however step one (above) is omitted and the UDS calls the GetSendData routine either as a response to its own software or hardware timing mechanism.
Text Messages
SkyPipe Pro has the ability to accept text message from a special program called SkyPipe Message Manager. These messages can be incorporated into the status bar, observation log, data file header, session log, and chat windows of a Stand Alone/ Server or Client instance of SkyPipe. Messaging is useful for a number of reasons, but a good example, would be to notify all clients that you have changed frequency and to automatically note this information in your own observer log. The UDS can send messages to the same processing routine in SkyPipe used by the Message Manager program. Simply call the SendMsgToSkyPipe(M$) routine in the UDS with your message represented by M$. Details on excluding the message from various destinations is included in the code.
Printer Port Control
Some users may want to use the Printer Port control available through the SkyPipe program. This would only work when the UDS was located on the same PC as Radio-SkyPipe, but it saves the user having to add substantial code and potentially expensive controls to their program to perform this function. The Radio-SkyPipe printer port control (TVicPort.OCX) works for all Windows 95 and above operating systems. Many of the free controls available on the web cannot control printer ports on Win 2000 or Win Xp.
Presently, only four printer port related commands are available. The first command initializes the TVicPort OCX, and should be performed once before data collection begins. The next two commands set any particular pin to a High or Low state. (Note only certain pins can be controlled in this way on a printer port.) The final command can read the state of any particular pin. When this command is issued, SkyPipe will return either a PINH or PINL response. It is up to the user to write the UDS code in such a way that it waits for either a PINH or PINL to be returned. One method to do this might be to have an integer variable (with module wide scope), let's call it PinState. Before reading a pin via SkyPipe, set PinState to -1. Then in the Socket Read routine, you could set PinState to 1 if a PINH command is received or to 0 if a PINL command was received. Monitoring the value of PinState in a loop would then let you know when the answer to your query had been returned by SkyPipe.
Sound Card Readings
Radio-SkyPipe uses proprietary OCXs to acquire sound card readings. You
would have to purchase these OCXs from swiftsoft or create your own sound
reading routines in order to use sound as the data source in your UDS. To
avoid this stumbling block I have provided means to get the sound card peak and
peak average values using SkyPipe's interface to the sound card. Use the
PEAK and WAVG commands to get to retrieve these values from SkyPipe, then apply
your own code to filter or manipulate the data in any way you want before
sending the data back to SkyPipe for charting.
Command Summary
All commands are strings. Binary data is not used. All commands end with a single byte ASCII character 255. In the command list below, the quote characters are not actually sent and are present in the description just to show that this is a literal string.
Commands sent by SkyPipe to the UDS
"INIT"+Chr(255) - Run UDS initialization Routine
"POLL"+Chr(255) - Run UDS POLLinit Routine - SkyPipe will poll the UDS for data.
"PUSH"+Chr(255) - Run UDS PUSHinit Routine - SkyPipe expects the UDS to push data.
"GETC"+Chr(255) - SkyPipe is requesting the number of channels from the UDS.
"STAT"+Chr(255) - SkyPipe is requesting a UDS Ready message from the UDS.
"GETD"+Chr(255) - SkyPipe is requesting sample data from the UDS. This is used only in polling mode.
"PINH"+Chr(255) - Value requested for a printer port pin was High.
"PINL"+Chr(255) - Value requested for a printer port pin was Low.
"PEAK"+Space + Peak Right Channel Value + Space + Peak Left Channel
Value + Chr(255)
"WAVG"+Space + Average Right Channel Value + Space + Average Left
Channel Value + Chr(255)
Commands sent by the UDS to SkyPipe
"^^1001" + Chr(255) - the UDS Ready message.
"^^1002" + Message String + Chr(255) - Sends a text message for processing as a would be done via Message Manager
"^^2013" + single character string representing the number of channels + Chr(255) - the number of channels
"^^3002" + timestamp string + Chr(255) - special timestamp used at the beginning of a push mode session.
"^^3000" + timestamp string + Chr(255) - update last sent data in chart using this time stamp
"^^3001" + Chr(255) - update last sent data in chart using SkyPipe generated timestamp
"^^4001" +Chr(255) - initialize the printer port I/O control.
"^^4002" + printer port pin number + Chr(255) - set printer port pin High.
"^^4003" + printer port pin number + Chr(255) - set printer port pin Low.
"^^4004" + printer port pin number + Chr(255) - read printer port pin. Value returned as PINH or PINL command.
"^^5001" + Chr(255) - Start the Sound Input
"^^5002" + Chr(255) - Stop the Sound Input
"^^5003" + Chr(255) - Get Peak R and L Sound
"^^5004" + Chr(255) - Get Average R and L Sound
"#"+ single character string representing channel number + Data String + Chr(255) - This is data for a given channel
Example UDS Projects
Several Example UDS projects are available. These projects include VB6 source code. These projects may also run under VB5.
Arduino UDS Project - A serial based UDS
using the Arduino
platform.
UDS1.vbp - The skeleton UDS project. As it is, this project will generate pseudo-random data. This project could serve as the starting point for most any UDS project. Remember to make a blank backup of the project before modifying the code to suit your own needs.
RSDVMUDS.vbp - A functional UDS project that uses the Radio-Shack computer enabled Digital Multimeter as a data source. This UDS communicates with the meter via a serial port and thus includes the a Microsoft Comm Control Active X component. This code could be used as a basis for other projects using a serial port connected device.
MetexUDS.vbp - Almost identical to the RSDVMUDS project, this functional example works with the Metex 11 Multimeter. These meters are sometimes available for as little as $40 US.
FileUDS.vbp - Demonstrates how data could be retrieved from a text file. This provides a possible way of getting data from a DOS program. This method would only work for relatively slow data rates in real time.
MAXUDS.vbp - Win95/98/NT/2000/Xp compatible example running the MAX186 and
MAX187 ADCs through SkyPipe's interface to these devices.
MAXUDS_Win98.vbp - Interfaces to MAX186 and MAX187 ADCs on Win95/98/NT PCs. This example uses a free DLL that can be installed on a non-SkyPipe PC.
UDS_Sound - Demonstrates accessing the sound card by capturing sound peak and peak average levels from SkyPipe's sound card interface.
SpectraCyber_UDS - Will work with Radio Astronomy Supplies SpectraCyber radiotelescope. It will also work with there 40 kHz VLF reciever that shares the same data format.
C++ UDS Example - Thanks to Dave Halley for contributing his C++ UDS source code which can serve as a basis for building other C++ UDS programs.
WinRadioUDS - Select up to 8 frequencies and sequentially tune the radio to each frequency, sample the signal level and return the values in separate channels.
RX320 - Same as WinRadioUDS except for the TenTec RX320 receiver.
Dataq Instruments UDS - Support for dataq.com 's line of ADCs. Tested only on
the inexpensive 194RS.
SDR14_UDS - Experimental UDS for the RFSpace SDR14 software defined receiver.
Pico ADC10 and ADC12 UDS - Support for these ADCs from Pico.com .
There are some others included in various states of completion.
Download the Available example code and executables at here.
Submitting UDS code and executables
The success of the UDS model relies largely on contributions of UDS executables and source code from contributers. After I receive and review a UDS submission, I will place it on the website where it can be downloaded by those who need it. The executable of the UDS should be submitted as some users will not have the ability to compile code. It is hoped that contributers will also share their source code to make modifications easy by others.
I understand that there is a wide range of coding styles in use, however, when
writing for others, it is very important that you try to make your code as
readable as possible. Please use comments freely, make variable names
understandable, and follow the rules of good indentation.
In some cases, I may be able to help you with your UDS need, especially if there is sufficient interest in the particular device to make the make the UDS useful to a number of people. I cannot be expected, however, to program for a device that I do not have access to. I hope that people who submit UDS executables and source code will be willing to help when problems arise in their implementation. This is usually the case.
