In this entry I will provide the latest versions of test programs I use to check the real-time performance of threads on soft real-time operating systems such as Windows, Linux and Mac OSX. Soft real-time systems can provide near true real-time performance, but can have delays in their response to external signals of 0.1 seconds or longer depending on how they are used. This topic was previously covered in a much older entry where I was testing the performance of the Qt Framework for use in real-time programming. In a future entry we will have a look at how the hardware you use can affect performance.
Download Code and EXE Files in ZIP format
Qt source code can be obtained here (29 KB). You can open the .PRO files in Qt Creator and build on Windows, Linux or Mac using Qt 4.8 to 5.2. The .PRO file can also be imported into Visual Studio if properly configured with Qt. These programs were tested with Qt 4.8 and 5.2 on Win 7 and VS2012, Qt 5.2 on Mac OSX 10.9.3 and Linux distribution Knoppix 7.2.0 which includes Qt 4.8 command line tools.
Ready to run Windows EXE files with Qt 4.8 can be obtained here (5.5 MB).
Four areas of testing will be covered: (1) soft real-time thread processing, (2) database record insert and read speed, (3) TCP and UDP transfer speed and (4) proper use of Qt Threads (discussed in a future post).
Soft Real-Time Process Testing
The Qt_RealtimeIO_App application runs a high priority thread on periodic intervals using various methods to control the intervals. In a future topic you will find some examples of real tests showing how hardware can affect the results. Note that running this test in a virtual machine will give very poor results.
While all these programs can run at user level, this one makes use of administrator or root level OS features and needs to run at root level for accurate results.
On Windows, right click the EXE program and use the Run as Administrator option.
On Linux or Mac, open a terminal window, move to the directory the program executable is in and enter the command:
sudo ./QT_RealtimeIO_App
Here are how the controls can be set.
Clock Mode
QTimer Event: uses a QTimer object in periodic mode to fire timer events. Clock periods of zero are invalid in this mode.
Qt msleep: uses the msleep function to perform timer waits. A clock period of zero can be set to cause the thread to wait for and time UDP packets being sent by another computer.
Win32 Periodic (Windows only): uses a win32 periodic waitable timer for timing. This is the most accurate way to do it on Windows. Older versions of Windows may provide timing 1 millisecond longer than the set rate. Clock periods of zero are invalid in this mode.
RTC wait (Linux only): if this option is enabled in the kernel, Linux provides the ability to use the Real-Time-Clock interrupt as a waiting signal. Only power of 2 millisecond periods are allowed (1, 2, 4, 8, etc.). Incorrect rates will be adjusted to next lowest valid value. A period of zero is invalid in this mode.
Clock Rate (msec)
Set the millisecond clock rate that you want to run. This can be set to zero when using the QTimer mode for testing UDP packet input response.
Send UDP
Set the IP address of a computer on your network and check this to send periodic UDP packets to the other computer. The program can be run at a zero period in QTimer mode at the other computer to time the input of packets.
Actual Rate
Shows the average timer period to 0.0001 seconds accuracy using the OS system clock.
Process Level (Windows only)
Sets the process priority on the Windows platform. You must be running as Administrator to use the Realtime level.
Thread Level
Sets the Qt or Windows thread level. Some high levels will only work in Administrator or root level. When running as root or superuser on Linux and Mac it is possible to use the UNIX SCHED_FIFO Maximum level which provides near real-time performance.
Control Buttons
The Start and Stop buttons control the running of the single periodic thread.
Message Area
If you are not running as Administrator or root, you may see error messages here when the program starts. The program attempts to lock the program in memory (prevent swapping) when running on UNIX style systems.
When running the periodic thread, you will see a histogram showing the number of times and percentage of times the thread ran at a specific period. Timing errors are sometimes bimodal where one sample will be long but the next will be short to compensate. The period should cluster near the set rate. Should the list become long due to variations, you can expend the dialog window in the vertical direction to steady the output. All timing uses the OS high resolution system clock which can possibly introduce some errors of its own.
You can run other processes and perform GUI operations while the program is running to see how it affects real-time thread processing. For example, right now I’m running QTimer mode on a Mac at 1 ms periods at user level. The peak period is 0.0011 seconds with 86% of the counts, but because my machine is heavily loaded at the moment, one period took 0.3956 seconds. The rest of the times vary between 0.0 and 0.0018 seconds.
Database Speed Testing
The Qt_SQL_App application uses QSqlQuery to test record insertion and reading rates in standard databases. To use it, set these controls. The program will create two tables named Table1 and Table2 so make sure they won’t write over any tables you have defined. The SQLite database is compiled into Qt and will function on all systems.
All operations in this program are run off the idle loop of a QTimer. This means the program will perform operations when there are no other operations necessary (responding to buttons or writing to the screen). This provides fast operation but makes the program responsive to user input.
Note that this is a very simple test program and doesn’t use transactions or do anything fancy. By using certain features of the Qt database objects, you may be able to improve performance above what you will see in this program.
In general, here are my experiences with the different databases:
MySQL
Was very fast when using the more simple non-transactional engine in older versions. The latest versions provide performance similar to MSSQL. The older Windows MySQL V3 to V4 can easily corrupt data on a power failure from my experience. I don’t have experience with newer versions. From what I’ve been told, the Linux version is more robust.
MSSQL
This is the database I use when I need network or multi-user access. Microsoft’s database is only available on Windows but is a strong performer on that platform. I haven’t seen any corruption problems with this database on power failures (yet). Documentation doesn’t provide simple guidance on what to do if there is corruption. Like many things with Windows, the details are sometimes hidden away from prying eyes. The 10 GB data file limit of the express (free) version is reached very quickly. MySQL seems to store data in a more compact format.
One of the best small scale databases you can use if you don’t need multi-user or network access. Used in many other products and very robust in my experience.
PostgreSQL
I haven’t used PostgreSQL much so can’t provide much guidance. This test is quite old and may not work with newer versions. Qt support for this database is not tracking the latest versions. That is the reason I disabled direct access and used ODBC access. One advantage with PostgreSQL is the fact that it is fully free for commercial use unlike MySQL or MSSQL.
DB Type
MySQL: operates using the MySQL drivers on your system. You will need to install MySQL and provide access to libmysql. Set Host, User and Password to access the database. The test schema area will be used to create table names Table1 and Table2.
ODBC MySQL: set Host, User and Password to connect to a MySQL ODBC connection named test.
ODBC MSSQL: set Host, User and Password to connect to a MSSQL ODBC connection named test.
SQLite Memory: creates SQLite tables in memory for testing. This provides very fast performance but data is lost on program exit. Host, User and Password do not need to be set.
SQLite File: creates a SQLite file named test.db in the same folder as the program for testing. You must have write access to the folder you run the program in. A special SQLite mode is used which provides faster performance at the expense of potential loss of data on a power failure (loss of ACID level protection). Host, User and Password do not need to be set.
PostgreSQL: set Host, User and Password to connect to a PostgreSQL ODBC connection named test.
Open DB and Close DB
Press these buttons to open or close the database. If it doesn’t open an error will appear in the text list to the right.
Empty Table Buttons
These will empty the tables of all records when pushed. This may take a while and make the program not respond when doing so.
Drop Table Buttons
These will drop and recreate the tables. All records will be lost.
Table 1 and Table 2 Tabs
These program tabs will use a QSqlQueryModel to show the records in each table. They can be selected at any time.
Run Statistics
These fields show the total number of Operations, the Operations per Second and the current Auto Index being processed.
Operation
These radio buttons allow you to select various operations:
- None: the program waits in an idle loop waiting for you to select an operation.
- Sequencial Read: sequentially reads any records out of Table1 as fast as possible. The QSqlQuery object will normally cache the records and operations may run very fast once the records are passed through the first time.
- Insert Records: inserts records into Table1 as fast as possible. WARNING: you should not insert more than about 100,000 records (visible at AutoIdx) or you may cause heavy use of memory and program swapping.
- Copy Records 1 to 2: copys all the records out of Table1 and into Table2. When finished, enters the None operation state.
TCP/UDP Speed Testing
The Qt_TCP_ClientServerApp application makes full use of Qt’s TCP and UDP networking components in full Signal/Slot mode to test networking speed. On a Linux system, it can easily fill an Ethernet cable with 1 Gbs of data using very little CPU effort. On Windows 7 you will only get around 0.8 Gbs of performance and the CPU use will be higher. To test networking speed, you need two computers connected to an IPv4 network. When the network supports full duplex operation, you can check Send Data on both ends to transfer data in both directions.
Rec Mode
Set receive mode to either Binary Stream (TCP) or UDP on both computers. The Reserved setting was reserved for TCP text mode but never implemented.
WARNING: the Binary Stream will generate lots of network traffic and should be used with caution on any network with critical business or industrial equipment. If you flood the network with packets and shut down other computers, there may be potential for loss of service, equipment damage or threats to human safety.
Server Computer Operation
- Run the application.
- Select the Rec Mode.
- Press the Server button.
- When connected, check Send Data to send packets to the remote client.
Client Computer Operation
- Run the application.
- Select the Rec Mode same as the server.
- Set the IPv4 address of the server.
- Press the Client button.
- Messages should appear showing a connection at both ends.
- When connected, check Send Data to send packets to the server.
During Operation
About every 5 seconds each receiving system will display the total number of bytes received and the bytes per second. If errors occur, they will also be reported. A simple check is made of the UDP received data to verify it was correctly received. There are no checks on the TCP data, but the network stack should take care of that for us.
Testing Qt Thread Operation
To be continued in the next post.