Microsoft Mobile Nokia Lumia SensorCore SDK introduction and hands-on experience

Microsoft Mobile Nokia Lumia SensorCore SDK introduction and hands-on experience

    As early as this year's BUILD conference, Nokia announced SensorCore and some of its demos. Today, it is finally available. You can go to Building Apps for Windows to view the specific introduction, or you can go to Nokia Developer to view it. To download SensorCore SDK, you can go to the nuget page: Lumia SensorCore SDK 0.9.1.3 .

1. What is SensorCore SDK?

    SensorCore SDK is a set of APIs that can access various sensor data (acceleration sensor/digital compass/gyro) and location information on the mobile phone. This information can be used to track the user's body and exercise status. Generally, the sensor runs in the background with low power consumption and saves the data in the last 10 days. At present, the Health & Fitness application on the market has integrated the SensorCore SDK, which mainly uses the step counter function. The interface is shown in Figure 1 below.

Figure 1: The step counter function page of the Health & Fitness app

    Furthermore, it also provides statistics and analysis of data, as shown in Figure 2 below.

Figure 2: The statistical analysis page of the Health & Fitness application

    In fact, the Health & Fitness application only uses the Step Counter API in the SensorCore SDK. It also includes the Activity Monitor API, Place Monitor API, and Track Point Monitor API, which we will introduce in a later article.

2. How to enable SensorCore function

    In order to use the API provided by SensorCore , we need to make some settings in the functions of the mobile phone, specifically, to enable the Location and motion data functions, as shown in Figure 3 below.

Figure 3: Settings page required to enable SensorCore

    If we do not want to disclose the user information, we can turn off this function, or manually clear the motion data. In addition, developers must deal with the "user disables Location and motion data", because SensorCore will not work properly if either of these two is disabled .

3. Introduction to SensorCore API

    SensorCore SDK contains APIs including Step Counter API, Activity Monitor API, Place Monitor API and Track Point Monitor API.

3.1 Step Counter API

    The API provides information on the number of steps and distance the user walks or runs. Of course, if the user puts the device in the trouser pocket to ride a bicycle or ride in a car, then the step-counting function of the API may be inaccurate.

3.2 Activity Monitor API

    The API provides status information about the user's physical activity, for example, when the user started walking and when the user stopped walking. Of course, in the process of user activity conversion, the API will have a detection delay of about 5-10 seconds, mainly to filter out noise and reduce the possibility of false alarms. If the user puts the device in his trouser pocket to ride a bicycle or ride in a car, or just sway the device in his hand, false alarms will occur. Activities can be divided into the following categories:

  • Idle: The device is placed in an environment away from the human body such as a table.
  • Stationary: The user carries the device with him, but in a static state.
  • Moving: It contains two states: Walking or Running.

    If the application monitors some of the user's status, the application will receive this notification when the user's status changes.

3.3 Place Monitor API

    The API provides the geographic location information of the user staying for more than a certain period of time. It runs in the background. In order to make the power consumption as low as possible, it mainly uses the mobile phone base station and WiFi access point information to locate. Therefore, it will not actively activate GPS for geographic location tracking, unless other applications are already using GPS (such as navigation applications). Since this type of API works passively, the geographic location information it provides is not real-time. In addition to providing relevant geographic location information, it also attempts to infer the geographic location of the user's home and work.

    When the user brings the device to a certain place for more than 10 minutes, the place will be considered as a Known Places and will be added to the list of Known Places. Generally speaking, the radius of a single Known Place is 200 meters. The distance between two Known Places generally requires more than 500 meters. Therefore, even if there are two different geographic locations within 500 meters, the API will merge these locations into one Known Place. This is something that developers need to consider.

    Place Monitor will try to classify one place as "Home" and another place as "Work". The classification rules are as follows:

  • The period of time the device stays in a Known Place
  • How often the device moves and the user's behavior status
  • How often the equipment is used
  • How often the device is charged

    Based on the above rules, the classification processing program will generally give an inference of "Home" and "Work" within 2-3 days. If the user changes the work unit, the API will also modify the inferences of "Home" and "Work" after a period of time.

    Place Monitor will dynamically update the list of Known Places. If a known Place is no longer accessed, the Known Place will be removed from the list after a period of time.

    All Known Places will have the following attributes:

  • ID: A unique identifier for a given location.
  • Kind: It can have the following three values, frequent/home/work.
  • Position: Geographical location information.
  • Radius: The radius centered on Position.

3.4 Track Point Monitor API

    This API provides specific information about the user's movement. It is similar to the Place Monitor API, but instead of tracking the Home and Work locations, it tracks the path of the user's actions. Under normal circumstances, Track Point Monitor API is also passive tracking, that is, using the mobile phone base station and WiFi information to locate. If there are other applications using GPS, Track Point Monitor API will use GPS information for positioning.

    Track points are updated every 5 minutes at most, and the distance between two track points is required to be greater than 500 meters. If the user stays within 500 meters for a long time, then the API will consider it as the same Track Point and no update operation will be performed. The accuracy of Track Point depends on the accuracy of location information, that is to say, the accuracy of Track Point is relatively high in places where cell phone base stations and WiFi access points are dense. If the user is on an outing in a remote place and the GPS is not turned on, the Track Point may be updated very slowly because the mobile phone is always within the coverage of the same base station.

    Track Point Monitor API provides the following information:

  • Heading: The user's direction of action recorded by Route Point. This parameter is only available when GPS is turned on.
  • LengthofStay: The length of time the device stays in a place.
  • Position: Location information of Route Point.
  • Radius: The radius of the location, which is related to the accuracy of the location.
  • Timestamp: The time point when the user entered the location.

4. SensorCore development environment requirements

For application development     using SensorCore , the required development environment is: Visual Studio 2013 Update 2 with Windows Phone SDK. SensorCore SDK supports simulator debugging, but the supported functions are very limited. Therefore, if you actually develop an application, it is recommended to debug it on an unlocked development device, such as the recently released Lumia 630.

5. SensorCore instance demo

    Let's take the Windows Phone 8.1 project as an example to demonstrate how to use the SensorCore SDK.

5.1 New project

    1. create a new project in Visual Studio, select

    Installed> Templates> Visual C#> Store Apps> Windows Phone Apps> Blank App (Windows Phone) and at the same time, name the project as HelloSensorCore.

5.2 Add SensorCore SDK to the project

    1. Select Tools> NuGet Package Manager> Manage NuGet Packages for Solution

    2. Search for Lumia SensorCore SDK and click Install.

    3. If you want to debug in the simulator, you also need to add Lumia SensorCore SDK Testing Tools . The operation steps are the same as the Lumia SensorCore SDK .

    4. After adding, you can check the References directory of the project and find that there are Lumia.Sense, Lumia.Sense.Testing and Microsoft Visual C++ 2013 Runtime Package. As shown in Figure 4 below.

Figure 4: Project add reference page

5.3 Add relevant content to the Package.appxmanifest file

    In order to enable the application to access the SensorCore API, we need to add related capabilities to the Package.appxmanifest file. If it is by adding a reference, the development environment will automatically add the following statement:

<DeviceCapability Name="location"/> 
    <m2:DeviceCapability Name="humaninterfacedevice"> 
        <m2:Device Id="vidpid:0421 0716"> 
        <m2:Function Type="usage:ffaa 0001"/> 
        <m2:Function Type="usage:ffee 0001"/> 
        <m2:Function Type="usage:ffee 0002"/> 
        <m2:Function Type="usage:ffee 0003"/> 
        <m2:Function Type="usage:ffee 0004"/> 
    </m2:Device> 
</m2:DeviceCapability>

    In addition, in order to make the program run correctly, we need to configure the target platform in Configuration Manager. If we test in the actual device, then we must choose ARM; if we test in the simulator, then we must choose x86. As shown in Figure 5 below.

Figure 5: Debugging platform configuration page

5.4 Use SensorCore API in code

    We create a very simple application here. The Grid element of the main page contains a ListBox to display the data returned by the SensorCore. Add the following code in the MainPage.xaml file:

<Grid> 
    <ListBox x:Name="SensorcoreList"/> 
</Grid>

    In the background code file MainPage.xaml.cs, first add the namespace reference:

using Windows.UI.Popups; 
using Lumia.Sense; 
using Lumia.Sense.Testing;
using System.Threading.Tasks;

    Add the following private variables in the MainPage class:

private PlaceMonitor _placeMonitor; 
private RouteTracker _routeTracker; 
private ActivityMonitor _activityMonitor; 
private StepCounter _stepCounter;

    Next, add Loaded event handling code in the constructor of MainPage:

this.Loaded += async (oo, ee) => 
{ 
    await ShowStepCounter(); 
    await ShowActivityMonitor(); 
    await ShowRouteTracker(); 
    await ShowPlacesMonitor(); 
};

    Then, process the activation and deactivation of SensorCore according to the visibility of the application:

Window.Current.VisibilityChanged += async (oo, ee) => 
{ 
    if (!ee.Visible) 
    { 
        if (_placeMonitor != null) await CallSenseApiAsync(async () => await _placeMonitor.DeactivateAsync()); 
        if (_routeTracker != null) await CallSenseApiAsync(async () => await _routeTracker.DeactivateAsync()); 
        if (_activityMonitor != null) await CallSenseApiAsync(async () => await _activityMonitor.DeactivateAsync()); 
        if (_stepCounter != null) await CallSenseApiAsync(async () => await _stepCounter.DeactivateAsync()); 
        } 
    else 
    { 
        if (_placeMonitor != null) await CallSenseApiAsync(async () => await _placeMonitor.ActivateAsync()); 
        if (_routeTracker != null) await CallSenseApiAsync(async () => await _routeTracker.ActivateAsync()); 
        if (_activityMonitor != null) await CallSenseApiAsync(async () => await _activityMonitor.ActivateAsync()); 
        if (_stepCounter != null) await CallSenseApiAsync(async () => await _stepCounter.ActivateAsync()); 
    } 
};

Define two methods to check whether the device supports SensorCore SDK, and check whether Location and motion data have been opened. Generally speaking, users will turn off these two options by default, so we need to provide users with a quick setting method instead of looking for these two options in the settings.

private async Task<bool> CallSenseApiAsync(Func<Task> action) 
{ 
    Exception failure = null; 
    try 
    {
        await action(); 
    } 
    catch (Exception e) 
    { 
        failure = e; 
    }     
    if (failure != null) 
    { 
        switch (SenseHelper.GetSenseError(failure.HResult)) 
        { 
        case SenseError.LocationDisabled: 
            await CreateMessageDialog("Location has been disabled. Do you want to open Location settings now?", "Information", "Yes", async cmd => await SenseHelper.LaunchLocationSettingsAsync(), true); 
            return false; 
        case SenseError.SenseDisabled: 
        await CreateMessageDialog("Motion data has been disabled. Do you want to open Motion data settings now?", "Information", "Yes", async cmd => await SenseHelper.LaunchSenseSettingsAsync(), true); 
            return false; 
        default: 
            await CreateMessageDialog(SenseHelper.GetSenseError(failure.HResult).ToString(), "Failure", "OK", null, false); 
        return false; 
        } 
    } 
    return true; 
}
private static async Task CreateMessageDialog(string message, string title, string label, UICommandInvokedHandler command, bool no) 
{ 
    var dialog = new MessageDialog(message, title); 
    dialog.Commands.Add(new UICommand(label,command)); 
    if (no) dialog.Commands.Add(new UICommand("No")); 
    await dialog.ShowAsync(); 
}

    Then we can start using StepCounter. CallSenseApiAsync encapsulates a method for securely accessing the SensorCore SDK. In this method, we first instantiate a StepCounter object and call StepCounter.IsSupportedAsync() to confirm whether the current device supports StepCounter. Then you can get the data related to the current StepCounter and display it in the Listbox on the main page.

private async Task ShowStepCounter() 
{ 
    await CallSenseApiAsync(async () => 
    {
        if (_stepCounter == null) 
        { 
            _stepCounter = await StepCounter.GetDefaultAsync(); 
        } 
        if (await StepCounter.IsSupportedAsync()) 
        { 
            var reading = await _stepCounter.GetCurrentReadingAsync(); 
            SensorcoreList.Items.Add("Current step counter reading"); 
            if (reading != null) 
            { 
                SensorcoreList.Items.Add(reading.Timestamp.ToString()); 
                SensorcoreList.Items.Add("Walk steps = "+ reading.WalkingStepCount); 
                SensorcoreList.Items.Add("Walk time = "+ reading.WalkTime.ToString()); 
                SensorcoreList.Items.Add("Run steps = "+ reading.RunningStepCount); 
                SensorcoreList.Items.Add("Run time = "+ reading.RunTime.ToString()); 
            } 
            else 
            { 
                SensorcoreList.Items.Add("data not available"); 
            } 
        } 
    }); 
}

    Next, we add ActivityMonitor to display the current user status. The processing flow is similar to the StepCounter above, instantiate an ActivityMonitor object, check whether the device supports it, and then get the current value and display it in the Listbox. The following are the RouteTracker and PlacesMonitor parts, the processing method is similar to the above, and will not be listed here. You can refer to the attached source code project. The main page of the demonstration is shown in Figure 6 below.

Figure 6: Application debugging main page information

    In addition, the SDK also provides several demos, including Steps, Activities, Places and Tracks. The effect of the demonstration is shown in Figure 6 below. You can go to Nokia Developer to download: link .

    The online document link provided by the SDK is as follows: Document link

    Next, Lumia App Labs webinar will provide training on how to use SensorCore SDK, the registration address is: training video registration link

Enjoy!

Reference link:

1. Building Apps for Windows

2. Lumia SensorCore SDK 0.9.1.3

3. Nokia Developer

4. Links to online documents

5. Training video registration link

6. Engineering source code

Reference: https://cloud.tencent.com/developer/article/1017654 Microsoft Mobile Nokia Lumia SensorCore SDK introduction and hands-on experience-Cloud + Community-Tencent Cloud