Collected data-- ideal collection rule interval?

MR
Matt Rogers
GfK Mediamark Research & Intelligence, LLC

I have been having an issue that I hope the minds here can help with.

We are using Mobicontrol Cloud and have been having issues since version 11 (we are now on 14.1) in which we find that we don't reliably get all the specified data from each device, even when they are online continuously, and I think I know why but would like ot hear your thoughts and opinions.

We are probably trying to hard to get as much data as possible instead of just what is needed (of all the data we can specify in the collection rule we ope to NOT select only 4 parameters, and I think we are trying to grab it too frequently (20 minute interval). It seems like the 20 minute interval may not be enough to upload each manifest before the the schedule starts collecting again, this assumes that the collection is set up as a round-robin: connect to Device A, upload all that is there to take, connect to Device B, gather that data...and so on. 

I think we may benefit from taking the following steps:

-Reduce the individual items that we collect to only what is needed

-Set up different collection rules and stagger them. Our single most important data to get is Location, so we could have a collection rule just for that and make sure it takes place on a schedule to avoid conflict with any other collection rules.

Does anyone know if each rule gets its own thread when it executes? I'd like to understand if we can collect the same data if we simply break it up into smaller chunks per rule/schedule, or if it is too much data to collect in the current time interval (20 minutes) and we need to consider going to 30 or 45 minute schedules.

Thanks!

7 years ago
Android
ANSWERS
RC
Raymond Chan Diamond Contributor
7 years ago (edited 7 years ago)

Could you please provide more information :

- How many devices are being managed by your MobiControl server?  What brand of devices?

- Apart from Location, what other items are the most important you need to collect data?

- Did you totally miss the expected data or did they just not arrive timely enough (i.e. too long delay before data already collected by the agent actually arrive the server)?  Is there any data item you need very timely (or near real-time)  collection completed on the MobiControl for action/consumption by other software/administrators?

- Are the devices mounted on cars? or used mostly in indoor environment?  Battery powered most of the time or powered via USB cables/cradles most of the time?

- What did you set for your device update schedule and the connection mode?

- What is the size and time you set for data truncation in the schedule tab of your Data-Collection rules?

MR
Matt Rogers
7 years ago

Hi Raymond, thanks for your reply

  • Approx. 200 Samsung android tablets, mostly Tab S2 with some Tab S3 and some Tab S4, all LTE/4G
  • Location is our biggest need, other must-haves are battery, storage, signal strength, network type
  • We sometimes seem to have skipped data collection for devices that are known to have been "on" continuously during the time we expect data; for example, if a device shows that it has been online for 10 hours it is not unusual to see that instead of having 100% data collection for a rule scheduled every 20 minutes  we will see gaps spanning multiple collection times-- an hour even, 40 minutes even, etc...whole multiples of the schedule interval.
  • The devices are not mounted in cars, they are handheld in a mix of indoor/outdoor environments. They are almost always on battery during the time they are in use but remain on even overnight, when they are connected to a charger (or are supposed to be). A device presumed to be at rest overnight and plugged into the charger will still have missed data collections
  • Connection mode is persistent, data collection is set to a 20 minute schedule
  • I have truncation set to 9999 and 190 days but the issue is that there are gaps that fall well within what we might expect to lose from truncation, and we have gaps for devices that have only been enrolled for a day so truncation would never come into play for those devices, at least not for some time
RC
Raymond Chan Diamond Contributor
7 years ago (edited 7 years ago)

Hi Matt,

With your answers or clarification to some of my questions, I think you have to consider the following:

- GPS location information may not be available due to sensitivity (qualtiy) of GPS hardware and ever-changing signal/noise levels received by the device.  In Hong Kong, where there are lots of skyscrapers everywhere, it is not uncommon for many of my Samsung test devices to fail getting GPS location in city/crowded areas.  We might get location when we take the device near the windows in the office of a tall building, and fail right away if we walk a 2-3 metres away from windows.   In general, the situation will be better  if you also turn on location option (A-GPS, etc.) to get coarser location information based on location of cellular base station.  Also, to be sure if missing location is due to unavailable information due to above-mentioned GPS related reasons,  I usually collect location together with battery level information in the same Data-Collection rule. 

- If you set the truncation parameters to 9999 KB and 190 days, they should be OK to avoid unexpected/unreasonable truncation.  In my previous tests,  setting 2048 KB or 4096 KB are usually enough for storing 2-3 items without truncation even if the device has been out-of-contact with server for 2 or more weeks.

- The shortest collection interval currently supported in MobiControl Data-Collection rules  is 2-minute.   There is no such thing as optimal schedule settings.  Just choose the interval as frequent as you need for each item of interest.  The major considerations are battery power and data truncation.  E.g., many battery-powered devices will have flat battery in about 2-4 hours if data collection of GPS location is scheduled every 2 minutes.  Getting battery/storage information alone every 2 minutes has much much smaller impact to battery life.  Set a bigger buffer to prevent truncation if the collection interval is short and more data-collection items are selected in the DC rule.

- As there is no direct script command or other ways to force flushing of data collected on device DC data buffer(s), delay of data can sometimes be an issue if very timely data is needed for some customers' use cases (e.g  near real-time tracking of lorry locations for a logistics company).  Even if the data collected is in the device buffer and device do sync updated rule/profile policies with the server after a forced device check-in by an administrator, data-collected might not be flushed and saved onto MobiControl server database.   My current workaround is to use a shorter "Update Schedule" (which defaults to 2-hour for any new device) in Advanced Settings.  This seems to help a bit for some device model.  One major drawback is increased cpu/bandwidth burden to the deployment server if there are lots of devices served by the server.

- Newer Android versions (6.x onwards) and different device OEM vendors have different ways to implement aggressive low-power modes (e.g. doze mode) to lengthen battery life.  As a result, MobiControl device agent may be forced to sleep and thus scheduled  data collection may not be possible.  If even battery-level information is missing frequently in your data-collection rule tests,  it is worthwhile to check if power-saving mode on your device is the cause.