Monday, November 20, 2017

Windows BLE analysis - a SensorTag approach

Windows Bluetooth system analysis – a SensorTag approach

In the following, I will do an analysis of the Windows operating system from the point of view of communication with Bluetooth Low Energy devices – in our case with SensorTags.

I will analyze Windows 7, Windows 8 and the following Windows 10 versions:
  • Anniversary Update (released on August 2, 2016; end of support Tentatively March 2018),
  • Creators Update (released on April 5, 2017; end of support Tentatively September 2018) and 
  • Fall Creators Update (released on April 5, 2017; end of support Tentatively September 2018).
The analysis will be done from the following points of view:
  1. The ability of the operating system to pair with the SensorTag; 
  2. The ability to get Generic Access data (it is a mandatory service); 
  3. The ability to get Device Information (this service exposes manufacturer and/or vendor information about a SensorTag);
  4. The ability to get the SensorTag's data using for this the reading approach and
  5. The ability to get the SensorTag's data using for this the notification approach.
All the tests will be done using version 9.7.8.0 of the blessTags application.


Windows 10 - Anniversary Update - Version 1607


This version of the Windows 10 operating system is the best one, from the point of view of Bluetooth Low Energy devices. It is able to pair without any problem with all SensorTags (regardless of the software version running on them), with which blessTags application knows how to work (CC2650STK, Thunderboard React, Thunderboard Sense and CC2541), and the data from the Bluetooth’s Services Get Generic Access and Get Device Information is acquired without any problem.
Analyzing the data acquisition speed using notifying and reading mechanism of data transfer, we can observe the following (these values were determined using the CC2650 device):
  1. through the notification mechanism, we can get data from all eight sensors from 150 ms to 150 ms without any problems;
  2. instead, when we set the acquisition time to 150 ms and we use the reading data acquisition mechanism - in the happiest situation, we get 713 ms and in the worst case, we get data from all eight sensors each 840 ms.
If we will analyze Thunderboard React and Thunderboard Sense we will get the similar results – they work without any problem in the Windows 10 Anniversary Update environment.
In fact, all the presentation movies of the blessTags application and features have been made with the support of the Windows 10 Anniversary Update.
A small demonstration and proof of the above information:

 

       Windows 10 - Creators Update - Version 1703


 The Creators Update version of Windows 10 is the worst operating system from the point of view of Bluetooth Low Energy devices. Nothing is working. Microsoft acknowledged that the Creators Update breaks Bluetooth Low Energy (reference1 and reference2). The Microsoft company promised a hotfix as soon as possible. But since then they have released an updated version of Windows (Fall Creators Update) and nothing has happened - within the Windows 10 Creators Update version, the Bluetooth Low Energy still does not work.
There are a large number of posts on forums in which different peoples complains regarding different types of Bluetooth devices that stop working after upgrading to Creators Update (see here, see here, see here, see here etc.).
The results, I'm going to show right away, were obtained after many tests: (1) on a desktop PC that had a CSR4.0 Bluetooth USB dongle (CSR8510 A10) and (2) on a Dell Inspiron P66F laptop with an integrated Bluetooth device. I know there are many solutions to fix several types of Bluetooth issues. I tried all, but nothing was working (update the Bluetooth driver, run Windows troubleshooter, disable and enable Bluetooth related services etc.)
So, let’s present the results:
1.      CC2650STK:
a.         On the firmware version 1.40 pairing the SensorTag device with Windows is impossible (I repeated the process several times, at least 8-10 times, I turned off and on the Bluetooth and I tried again – the results were the same: it was impossible to add this device).
b.           In the firmware version 1.20, the PC discovered the SensorTag and I was able to pair the SensorTag with the PC. 
    Also, I was able to get Generic Access data. But, at the Device Information service, from 9 characteristics only 6 responded and only from them it was possible to get data.
Instead, I cannot set up the device and I cannot retrieve data either through the read mechanism or through the notifications.
2.      Thunderboard React.
The operating system has a strange behavior when the pairing process is initiated. In the list of discovered devices, the SensorTag appear and disappear (with a period of 1 … 1.5 s). Finally, when a mouse clicks succeed on the SensorTag, the pairing process accomplishes and the LEDs on the Thunderboard React (the blue and the green ones) have a period when they are flashing consecutively in an atypical mode.
The reading of the characteristics of the Generic Access Service can be done without any problem, but the reading from Device Information Service fails on all four existing characteristics.
Setting the sensors, the mode of acquiring data (on Thunderboard React you have only the following possibility: (1) to get data through the notification from 3 sensors and (2) to read data from the other four sensors) is impossible. Therefore, the impossibility of obtaining actual data from sensors results directly from here.
3.      Thunderboard Sense.
The same pulsating process, observed for Thunderboard React, was found to be also existing for Thunderboard Sense - when we want to achieve the pairing process. But here, things are even worse: after pairing the blessTagprogram cannot detect the SensorTag. So, no active device - no entity from where the blessTags application to acquire the data.
4.      CC2541DK
The behavior is similar to the behavior of CC2650STK (firmware version 1.40). At each connection attempt, you will get the following error message: "Try connecting your device again".
So, in conclusion, within this version of Windows 10 (Creators Update), it is impossible to communicate with any of the four types of SensorTags point out above. Consequently, I mention (once again) that here I have used the same software version that I also used in all test made on Windows 10 Anniversary Update.

 

       Windows 10 – Fall Creators Update - Version 1709

 

     This version of Windows 10 (1709) is a huge step forward, compared with Windows 10 Creators Update (were on BLE almost nothing is working), but still has a long way to get to the level of Windows 10 Anniversary Update (1607) operating system.

But let's see why I made this statement:

1.      CC2650STK (firmware version 1.40) & CC2541DK:

I will treat these two devices here simultaneously because their behavior related to the Windows 10 (1709) operating system is similar.

The pairing operation and the reading from the Generic Access and the Device Information services are working perfectly without any kind of problems.

The problems only occur when we want to read information from the sensors. The data transfer mechanism through notifications does not work at all. The only way to get data from the sensors, embedded in the SensorTag, is by means of the direct reading mechanism from the device. This approach has two issues: (1) lower data transfer speed (as we have shown above) (2) if all sensors accept the two data transfer methods (through reading and notification), the buttons on the SensorTag can be interrogated only through the notification mechanism. Thanks to this "feature" of the Windows 10 (1709) operating system, the blessTags application implements, starting with version 9.7.8.0, the reading method for data acquisition.

A problem appears with the CC2650STK SensorTag having the firmware version 1.20. If the process of pairing and data reading from Generic Access service works very well, the reading process from Device Information services is not possible. Moreover, the sensors reading (from this SensorTag) does not work through either one of the two possible mechanisms (reading or notification).

2.      Thunderboard React

In the same mode as in Windows 10 Creators Update, the SensorTag appear and disappear when we want to add a new Bluetooth device. The same behavior can be highlighted in the action center on Bluetooth’s quick action button were "Not connected" and "Thunderboard React" are displayed repeatedly (please see the following movie starting from the time index 5.14). Immediately we can conclude that Thunderboard React is guilty, mainly due to a flawed implementation of the advertising mechanism by Silicon Labs engineers. But, searching on the internet, we will notice that other users reported this problem to other BLE devices after installing the Fall Creators Update – e.g. view this movie on youtube.

After pairing the SensorTag, the blessTags application is not able to find the Thunderboard React device. So, at this point nothing is working:  Generic Access and the Device Information services or data acquisition from the sensors embedded on Thunderboard React SensorTag.
3.      Thunderboard Sense
The mode to behave is similar with the one of Thunderboard React, this Bluetooth device is displayed and disappear repeatedly. When the pairing process succeeded, it is possible to take data from Generic Access Service. But from this point, nothing is working anymore.
As a conclusion, up now on Windows 10 Fall Creators Update (1709, build 16229.19) only the SensorTags produced by TI (CC2650STK and CC2541DK) are working. More, they are working only in reading mode. But attention! Only CC2650STK firmware version 1.40 will work in this mode. Unfortunately, when you buy a CC2650STK you have a very high chance of taking a device with firmware revision 1.20. So, to be able to communicate with a such a type of SensorTag an upgrade it is necessary at least to the firmware version 1.40.
A short demonstration, of the above statements, is presented below.

 

     Windows 8

Will be completed shortly.

 

     Windows 7

 
The Microsoft company has added support for the Bluetooth Low Energy (BLE) stack starting with the Windows 8 operating system. They have provided an API which enables applications to access BLE devices.
But the Microsoft has not ported the BLE API's to Windows 7. The Windows 7’s built-in stack supports only Bluetooth version 2.1/3.0, there is no support for BLE (4.0, 4.1 or 4.2). So, from the point a view of a developer it is impossible to communicate, in Windows 7, with a BLE device using Windows 7’s stack. 
The TI company have a program called the BLE Device Monitor that is able: (1) to run on Windows 7 and (2) to communicate with a SensorTag. But you must use for these a special USB dongle (e.g. CC2540 Bluetooth Low Energy USB). If the source code for the USB dongle is free, the source code for the BLE Device Monitor is not available – it is only for the internal use of the TI company.

 

 

Monday, October 23, 2017

Reading & notifying

Reading vs. notifying


Up to the version 9.6.7.0 of the blessTags application, the data transfer was carried out mainly, only where it was possible, through the notification mechanism. In this way, the battery on the SensorTag was saved and higher data transfer speeds were also achieved. 
Starting with blessTags application version 9.7.8.0 both reading and notifying data transfer was implemented, each time when a SensorTag allows us (when a characteristic support both mechanism of data transfer).

But what it is the significance of reading or notifying operations?

When a smartphone application or a PC application (like blessTags application) interacts with a SensorTag, over a Bluetooth connection, we have a client/server architecture. In our case, the application (e.g. blessTags application) is the GATT client and the SensorTag is the GATT server.


The hierarchical organization of the Services, Characteristics and Descriptors is presented in the above figure. In a Bluetooth device, a Service contains, at least, one or more Characteristics. A Characteristic owns none or more Descriptors. The Descriptors are completely optional, but a Service must contain at least one Characteristic.
In a Bluetooth SensorTag, part of the Characteristics are items of data which are related to a state of one sensor or of another sensor. Other Characteristics represent configuration data such as the period at which a sensor will measure a specific information – e.g. the ambient temperature.
Each Characteristic may have various properties. These properties define what operations are allowed or can be configured for the values associated with each Characteristic. Most common properties are:
  • Read (data transfer took place from server to client): if enabled, it will allow a client (an application) to read a value from the SensorTag (the server). 
  • Write (data transfer took place from client to server): if enabled, it will allow a client to change a value from the SensorTag. 
  • Indicate: if enabled, the client will be notified if a value changes (e.g. a new data was acquired) – in this case, a confirmation from the client to the server is expected. 
  • Notify: if enabled, the client will be notified by the SensorTag if a value belonging to a specific characteristic change – in this case, a confirmation is NOT expected.
Usually, a value may be read or write by specifying the characteristic's UUID.

The values associated with some characteristics could have multiple modes of data acquisition – e.g. through notifying and reading, like the temperature Characteristic of the CC2650STK and CC2541DK SensorTags, see the following two figures (the first one it is a print screen of the developer window of blessTags application and the second one is from Wikipedia official page of the CC2541DK SensorTag).


Information (service, characteristics and descriptors) obtained through the 
data developer functions of the blessTags applications

All these data were taken from the Wikipedia official page of the CC2541DK SensorTag User Guide
       Another Characteristic can only be accessed based only one method of data reading. One such example is the reading of the buttons states on the CC2650STK and CC2541DK that can only be read by means of the notification mechanism, see figures below (the first one it is a print screen of the developer window of blessTags application and the second one is from Wikipedia official page of the CC2650STK SensorTag). 
Information (service, characteristics and descriptors) obtained through the data developer functions 

Indications and Notifications provide a more efficient way for Clients to know when a value has changed.
The client (the application) may request a notification (or an indication) for a particular characteristic from the server (SensorTag) – the settings of this request must be configured by the client only one time at the beginning. The server can then send the value to the client whenever it becomes available or when a value has changed (instead of having to manually poll continuously and read the characteristic value to know when it changes). On each notification, your program (blessTags in our case) only get an event saying "you have received a packet" and in the associated function you will get the data packet.  For instance, a temperature sensor may notify its client every time it takes a new measurement. Using this approach, it will avoid the need for the client to continuously poll the server, which would require the server's radio circuitry to be constantly operational. Minimizing radio usage is especially important when developing systems that must work longer on a coin cell battery – like our SensorTag systems, mainly because radio usage has an adverse effect on a device’s battery life.
Notifications are also preferred if you need a higher data transfer rate between the server and the client. Using notifications, the server can queue a certain number of packets before sending – in this mode a higher data transfer ration can be achieved.

Windows Bluetooth system analysis – a SensorTag approach


In the following, I will do an analysis of the Windows operating system from the point of view of communication with Bluetooth Low Energy devices – in our case with SensorTags.

I will analyze Windows 7, Windows 8 and the following Windows 10 versions:
  • Anniversary Update (released on August 2, 2016; end of support Tentatively March 2018),
  • Creators Update (released on April 5, 2017; end of support Tentatively September 2018) and 
  • Fall Creators Update (released on April 5, 2017; end of support Tentatively September 2018).
The analysis will be done from the following points of view:
  1. The ability of the operating system to pair with the SensorTag; 
  2. The ability to get Generic Access data (it is a mandatory service); 
  3. The ability to get Device Information (this service exposes manufacturer and/or vendor information about a SensorTag);
  4. The ability to get the SensorTag's data using for this the reading approach and
  5. The ability to get the SensorTag's data using for this the notification approach.
All the tests will be done using version 9.7.8.0 of the blessTags application.


Windows 10 - Anniversary Update - Version 1607


This version of the Windows 10 operating system is the best one, from the point of view of Bluetooth Low Energy devices. It is able to pair without any problem with all SensorTags (regardless of the software version running on them), with which blessTags application knows how to work (CC2650STK, Thunderboard React, Thunderboard Sense and CC2541), and the data from the Bluetooth’s Services Get Generic Access and Get Device Information is acquired without any problem.
Analyzing the data acquisition speed using notifying and reading mechanism of data transfer, we can observe the following (these values were determined using the CC2650 device):
  1. through the notification mechanism, we can get data from all eight sensors from 150 ms to 150 ms without any problems;
  2. instead, when we set the acquisition time to 150 ms and we use the reading data acquisition mechanism - in the happiest situation, we get 713 ms and in the worst case, we get data from all eight sensors each 840 ms.
If we will analyze Thunderboard React and Thunderboard Sense we will get the similar results – they work without any problem in the Windows 10 Anniversary Update environment.
In fact, all the presentation movies of the blessTags application and features have been made with the support of the Windows 10 Anniversary Update.
A small demonstration and proof of the above information:

 

       Windows 10 - Creators Update - Version 1703


 The Creators Update version of Windows 10 is the worst operating system from the point of view of Bluetooth Low Energy devices. Nothing is working. Microsoft acknowledged that the Creators Update breaks Bluetooth Low Energy (reference1 and reference2). The Microsoft company promised a hotfix as soon as possible. But since then they have released an updated version of Windows (Fall Creators Update) and nothing has happened - within the Windows 10 Creators Update version, the Bluetooth Low Energy still does not work.
There are a large number of posts on forums in which different peoples complains regarding different types of Bluetooth devices that stop working after upgrading to Creators Update (see here, see here, see here, see here etc.).
The results, I'm going to show right away, were obtained after many tests: (1) on a desktop PC that had a CSR4.0 Bluetooth USB dongle (CSR8510 A10) and (2) on a Dell Inspiron P66F laptop with an integrated Bluetooth device. I know there are many solutions to fix several types of Bluetooth issues. I tried all, but nothing was working (update the Bluetooth driver, run Windows troubleshooter, disable and enable Bluetooth related services etc.)
So, let’s present the results:
1.      CC2650STK:
a.       On the firmware version 1.40 pairing the SensorTag device with Windows is impossible (I repeated the process several times, at least 8-10 times, I turned off and on the Bluetooth and I tried again – the results were the same: it was impossible to add this device).
b.      On the firmware version 1.20, the PC discovered the SensorTag and I was able to pair the SensorTag with the PC. 
       Also, I was able to get Generic Access data. But, at the Device Information service, from 9 characteristics only 6 responded and only from them it was possible to get data.
Instead, I cannot set up the device and I cannot retrieve data either through the read mechanism or through the notifications.
2.      Thunderboard React.
The operating system has a strange behavior when the pairing process is initiated. In the list of discovered devices, the SensorTag appear and disappear (with a period of 1 … 1.5 s). Finally, when a mouse clicks succeed on the SensorTag, the pairing process accomplishes and the LEDs on the Thunderboard React (the blue and the green ones) have a period when they are flashing consecutively in an atypical mode.
The reading of the characteristics of the Generic Access Service can be done without any problem, but the reading from Device Information Service fails on all four existing characteristics.
Setting the sensors, the mode of acquiring data (on Thunderboard React you have only the following possibility: (1) to get data through the notification from 3 sensors and (2) to read data from the other four sensors) is impossible. Therefore, the impossibility of obtaining actual data from sensors results directly from here.
3.      Thunderboard Sense.
The same pulsating process, observed for Thunderboard React, was found to be also existing for Thunderboard Sense - when we want to achieve the pairing process. But here, things are even worse: after pairing the blessTagprogram cannot detect the SensorTag. So, no active device - no entity from where the blessTags application to acquire the data.
4.      CC2541DK
The behavior is similar to the behavior of CC2650STK (firmware version 1.40). At each connection attempt, you will get the following error message: "Try connecting your device again".
So, in conclusion, within this version of Windows 10 (Creators Update), it is impossible to communicate with any of the four types of SensorTags point out above. Consequently, I mention (once again) that here I have used the same software version that I also used in all test made on Windows 10 Anniversary Update.


       Windows 10 – Fall Creators Update - Version 1709


 This version of Windows 10 (1709) is a huge step forward, compared with Windows 10 Creators Update (were on BLE almost nothing is working), but still has a long way to get to the level of Windows 10 Anniversary Update (1607) operating system.
But let's see why I made this statement:
1.      CC2650STK (firmware version 1.40) & CC2541DK:
I will treat these two devices here simultaneously because their behavior related to the Windows 10 (1709) operating system is similar.
The pairing operation and the reading from the Generic Access and the Device Information services are working perfectly without any kind of problems.
The problems only occur when we want to read information from the sensors. The data transfer mechanism through notifications does not work at all. The only way to get data from the sensors, embedded in the SensorTag, is by means of the direct reading mechanism from the device. This approach has two issues: (1) lower data transfer speed (as we have shown above) (2) if all sensors accept the two data transfer methods (through reading and notification), the buttons on the SensorTag can be interrogated only through the notification mechanism. Thanks to this "feature" of the Windows 10 (1709) operating system, the blessTags application implements, starting with version 9.7.8.0, the reading method for data acquisition.
A problem appears with the CC2650STK SensorTag having the firmware version 1.20. If the process of pairing and data reading from Generic Access service works very well, the reading process from Device Information services is not possible. Moreover, the sensors reading (from this SensorTag) does not work through either one of the two possible mechanisms (reading or notification).
2.      Thunderboard React
In the same mode like in Windows 10 Creators Update, the SensorTag appear and disappear when we want to add a new Bluetooth device. The same behavior can be highlighted in the action center on Bluetooth’s quick action button were "Not connected" and "Thunderboard React" are displayed repeatedly (please see the following movie starting from the time index 5.14). Immediately we can conclude that Thunderboard React is guilty, mainly due to a flawed implementation of the advertising mechanism by Silicon Labs engineers. But, searching on the internet, we will notice that other users reported this problem to other BLE devices after installing the Fall Creators Update – e.g. view this movie on youtube.
After pairing the SensorTag, the blessTags application is not able to find the Thunderboard React device. So, at this point nothing is working:  Generic Access and the Device Information services or data acquisition from the sensors embedded on Thunderboard React SensorTag.
3.      Thunderboard Sense
The mode to behave is similar with the one of Thunderboard React, this Bluetooth device is displayed and disappear repeatedly. When the pairing process succeeded, it is possible to take data from Generic Access Service. But from this point, nothing is working anymore.
As a conclusion, up now on Windows 10 Fall Creators Update (1709, build 16229.19) only the SensorTags produced by TI (CC2650STK and CC2541DK) are working. More, they are working only in reading mode. But attention! Only CC2650STK firmware version 1.40 will work in this mode. Unfortunately, when you buy a CC2650STK you have a very high chance of taking a device with firmware revision 1.20. So, to be able to communicate with a such a type of SensorTag an upgrade it is necessary at least to the firmware version 1.40.
A short demonstration, of the above statements, is presented below.

 

     Windows 8

                                                                         .....................

     Windows 7

 
The Microsoft company has added support for the Bluetooth Low Energy (BLE) stack starting with the Windows 8 operating system. They have provided an API which enables applications to access BLE devices.
But the Microsoft has not ported the BLE API's to Windows 7. The Windows 7’s built-in stack supports only Bluetooth version 2.1/3.0, there is no support for BLE (4.0, 4.1 or 4.2). So, from the point a view of a developer it is impossible to communicate, in Windows 7, with a BLE device using Windows 7’s stack. 
The TI company have a program called the BLE Device Monitor that is able: (1) to run on Windows 7 and (2) to communicate with a SensorTag. But you must use for these a special USB dongle (e.g. CC2540 Bluetooth Low Energy USB). If the source code for the USB dongle is free, the source code for the BLE Device Monitor is not available – it is only for the internal use of the TI company.

 

Saturday, September 16, 2017

Solving sensors communication errors

Dealing with sensors communication errors

  There are situations when after configuring one or more sensors embedded inside a SensorTag, on the main user interface:
  1. the information presented "freezes" (from one or more configured sensors) – e.g. does not change its value, while the information from other sensors is displayed correctly, or
  2. even we activated correctly a sensor, the associated controls are dimmed – on the blessTags user interface a control is dimmed only when: (a) the related sensor is invalidated and do not send data to the blessTags application or when (b) that sensor is not embedded on the SensorTag.
 Even if it looks surprising, the problem is the low battery level. Previously I said surprisingly because the battery level can be, for example, 67% or even 71% - the value displayed on the user interface for CC2650STK SensorTag. By replacing the battery with a new one, you will notice that all the problems listed above will disappear. I suspect that these false levels, of battery power, are due to a problem with the battery state measurement algorithm for CC2650STK (at least in the Firmware version 1.40).
 If the same battery is measured on a ThunderBoardSense SensorTag, battery power levels are much lower, somewhere in the 5 - 12% range.
 To prove all the statements presented previously, please view the movie below.