Data import and export¶
Ichnaea supports automatic, periodic CSV (comma separated values) export of aggregate cell data (position estimates). Mozilla’s exports are available at https://location.services.mozilla.com/downloads
The data exchange format was created in collaboration with the OpenCellID project.
Records should be written one record to a line with CRLF (0x0D 0x0A) as line separator.
A value should be written as an empty field—two adjacent commas, for example—rather than being omitted.
The first five fields (radio to cell) jointly identify a unique logical cell network. The remaining fields contain information about this network.
The data format does not specify the means and exact algorithms by which the position estimate or range calculation was done. The algorithms might be unique and changing for each source of the data, though both Ichnaea and OpenCellID currently use similar and comparable techniques.
The fields in the CSV file are as follows:
Network type. One of the strings
UMTS(for WCMDA networks), or
Mobile Country Code. This is an integer.
505is the code for Australia.
For GSM, UMTS, and LTE networks, this is the mobile network code (MNC). This is an integer.
4is the MNC used by Vodaphone in the Netherlands.
For GSM and UMTS networks, this is the location area code (LAC). For LTE networks, this is the tracking area code (TAC). This is an integer.
For GSM and LTE networks, this is the cell id or cell identity (CID). For UMTS networks this is the UTRAN cell id, which is the concatenation of 2 bytes of radio network controller (RNC) code and 2 bytes of cell id. This is an integer.
For UMTS networks, this is the primary scrambling code (PSC). For LTE networks, this is the physical cell id (PCI). For GSM networks, this is empty. This is an integer.
Longitude in degrees between -180.0 and 180.0 using the WGS 84 reference system. This is a floating point number.
Latitude in degrees between -90.0 and 90.0 using the WGS 84 reference system. This is a floating point number.
Estimate of radio range, in meters. This is an estimate on how large each cell area is, as a radius around the estimated position and is based on the observations or a knowledgeable source. This is an integer.
Total number of observations used to calculate the estimated position, range and averageSignal. This is an integer.
Whether or not this cell is a position estimate based on observations subject to change in the future or an exact location entered from a knowledgeable source. This is a boolean value, encoded as either
1(for “changeable”) or
Timestamp of the time when this record was first created. This is an integer counting seconds since the UTC Unix Epoch of 1970-01-01T00:00:00Z.
1406204196which is the timestamp for 2014-07-24T12:16:36Z.
Timestamp of the time when this record was most recently modified. This is an integer, counting seconds since the UTC Unix Epoch of 1970-01-01T00:00:00Z.
1406204196, which is the timestamp for 2014-07-24T12:16:36Z.
Average signal strength from all observations for the cell network. This is an integer value, in dBm.
This field is only used by the OpenCellID project and has been used historically as a hint towards the quality of the position estimate.
Aggregate cell data can be imported into an Ichnaea instance. For the development environment:
Download a Differental Cell Export from Mozilla’s Download page. Do not extract it, but keep it in the compressed
.csv.gzformat, in the root of the repository.
In a shell in the app container, import the data:
$ make shell # Replace with the filename of the downloaded export file app@blahblahblah:/app$ ichnaea/scripts/load_cell_data.py MLS-diff-cell-export-YYYY-MM-DDTHH0000.csv.gz
This will import the cell data, then queue tasks to aggregrate cell areas and region statistics. It should take about a minute to process an 300kB export of 10,000 stations.
Importing a Full Cell Export is not recommended. This will fail due to unexpected data, and the development environment may require undocumented changes for the larger resource requirements of a full cell export.