Highlights from
OpenStreetMap Functions

  • ...
    get the x,y coordinates of unique nodes at road intersections
  • ...
    EXTRACT_CONNECTIVITY extract road connectivity from parsed OpenStreetMap
  • assign_from_parsed(parsed...
    assign from parsed osm structure
  • get_way_tag_key(tag)
    get tags and key values for ways
  • load_osm_xml(filename)
    load OpenStreetMap XML file contents as a MATLAB structure
  • main_mapping(varargin)
    handle application map
  • parse_openstreetmap(opens...
    PARSE_OPENSTREETMAP parse an OpenStreetMap XML file (OSM XML)
  • parse_osm(osm_xml)
    PARSE_OSM Parse into a structure a loaded OSM XML structure.
  • plot_nodes(ax, parsed_osm...
    plot (selected) nodes and label each with index and id
  • plot_road_network(ax, con...
    plot the nodes and edges connecting them
  • plot_route(ax, route, par...
    plot (over map) the route found by route planner
  • plot_way(ax, parsed_osm, ...
    PLOT_WAY plot parsed OpenStreetMap file
  • route_planner(dg, S, T)
    find shortest path in graph of road intersections (nodes)
  • show_map(ax, bounds, map_...
    plot raster map in figure and fix plot bounds
  • debug_openstreetmap.m
    example script for using the OpenStreetMap functions
  • usage_example.m
    example script for using the OpenStreetMap functions
  • View all files

4.54545

4.5 | 11 ratings Rate this file 122 Downloads (last 30 days) File Size: 16.1 KB File ID: #35819
image thumbnail

OpenStreetMap Functions

by

 

22 Mar 2012 (Updated )

Load map, extract connectivity, plot road network & find shortest paths from OpenStreetMap XML file.

| Watch this File

File Information
Description

This software package includes functions for working with OpenStreetMap XML Data files (extension .osm), as downloaded from http://www.openstreetmap.org, to:
1) Import and parse the XML data file and store the parsed data in a MATLAB structure. This data represents the graph of the transportation network.

2) Plot the MATLAB structure to get a visualization of the transportation network, its nodes and their labels.

3) Extract the adjacency matrix of the directed graph representing the network's connectivity (i.e., road intersections).

4) Find shortest routes between nodes within the network. Note that distance is measured as the number of transitions between intersection nodes, not over the map.

Development and releases on GitHub:
   https://github.com/johnyf/openstreetmap
(linked also here).

An example map.osm is included to be used with usage_example.m

Required Products MATLAB
MATLAB release MATLAB 7.14 (R2012a)
Tags for This File   Please login to tag files.
Please login to add a comment or rating.
Comments and Ratings (43)
01 Aug 2014 Gummi

@shiva I had the same problem, I fixed it with a quick check in the for-loop in parse-osm.m :
for j=1:Nnd
if Nnd == 1
ndtemp(1,j) = str2double(waytemp.nd.Attributes.ref);
else
ndtemp(1, j) = str2double(waytemp.nd{j}.Attributes.ref);
end
end

As you see, the else part is what your line 78 looks like.

22 Jul 2014 shiva

first of all i have to thank you for your great package.
can you help me about why i get this error:
??? Cell contents reference from a non-cell array object.

Error in ==> parse_osm>parse_way at 78
ndtemp(1, j) = str2double(waytemp.nd{j}.Attributes.ref);

Error in ==> parse_osm at 37
parsed_osm.way = parse_way(osm_xml.way);

Error in ==> parse_openstreetmap at 43
parsed_osm = parse_osm(osm_xml);

Error in ==> usage_example at 29
[parsed_osm, osm_xml] = parse_openstreetmap(openstreetmap_filename);

thanks in advance

13 Jul 2014 pietro

Thanks Ioannis for your help,

so I need to zoom in the area and then I can analyze it through matlab, right? I have a huge gpx file and I need to create several papermaps. Can I do it with your functionset?

thanks

06 Jul 2014 Ioannis Filippidis

Just select "Export" at the top left and zoom in enough so that it won't complain on the export pane (opens on the left after you click "Export") that the area is too large to be exported. Then hit the blue "Export" button on the left pane, and that will download the ".osm" file that contains the XML data you need.

29 Jun 2014 pietro

Hi I can't find the OpenStreetMap XML Data file in the openstreet map website. Where can I find it?

thanks

25 Apr 2014 Ioannis Filippidis

gist for importing osm graphs as networkx graphs in python:
https://gist.github.com/aflaxman/287370/

07 Mar 2014 Ioannis Filippidis

Hi wes, thanks you for the review. If I recollect correctly, the ids should refer to the OSM database, so be global ids over the whole world, rather than indices in the connectivity matrix (which is a sample of the OSM database).

03 Mar 2014 wes

Hi Ioannis,

Great package you have here, working perfectly for what i need! thank you for taking the time to upload.

I am having a few problems with the route planner though. when i take the node id out of the parsed osm file i get a big number such as 1151088456 but i get an eorror message saying source node index invalid. if i randomly pick one of the small numbers out of the connectivity matrix i can get it working but i have no idea how to relate to these locations to there real world position untill i run it and see where they appear.

any advice on where i'm going wrong?

thanks in advance

23 Feb 2014 ghufran  
07 Dec 2013 Ioannis Filippidis

Also: http://stackoverflow.com/questions/11321718/how-can-i-display-osm-tiles-using-python

07 Dec 2013 Ioannis Filippidis

Please consider switching to Python. There are numerous OpenStreetMap facilities there and excellent visualization capabilities:

https://pypi.python.org/pypi?%3Aaction=search&term=openstreetmap&submit=search

23 Nov 2013 Ioannis Filippidis

Hi Gaurav,

I've updated the submission on github (and here, waits review) to fix the name shadowing conflict that presumably causes the "no osm field" error.

@Matthew:
the restorehold function is now included (rename was needed) and an example map.osm as well.

What was your reason for commenting that the "chosen XML parser is not appropriate for anything but the extremely simple example." ?

28 Oct 2013 gaurav

hey,
I have been trying to use this distribution to convert the region Dharavi,Mumbai, India to data set. However, I am not able to understand how to use the package and which is the main file which would call the matlab files given.
Also, I tried using the usage_example.m, however it shows error :
"Reference to non-existent field 'osm'.

Error in parse_openstreetmap (line 42)
osm_xml = map_osm.osm;

Error in usage_example (line 29)
[parsed_osm, osm_xml] = parse_openstreetmap(openstreetmap_filename);"

my input file was with name same as in example - map.osm and saved in the same directory as all other files in the distribution.
Your help would be highly valued as I am in urgent need for this thing to work

06 Oct 2013 Reham

Thanks so much for your reply.

06 Oct 2013 Ioannis Filippidis

The osm is loaded in MATLAB as a directed graph represented by an adjacency matrix. Assuming constant speed, travel time is proportional to distance. So for them to be different, different speeds need to be assumed for different types of roads. I would suggest assigning weights to roads (edges) depending on their size by storing the weights in the adjacency matrix. Then you can run Dijkstra's shortest path algorithm on the weighted graph.

Recommended reading the wikipedia article on Dijkstra's algorithm, which you can find implemented e.g. in http://www.mathworks.com/matlabcentral/fileexchange/10922-matlabbgl, as a starting point to learn more about the connectivity matrix you can look at the help text of extract_connectivity.m .

05 Oct 2013 Reham

Okay, Problem resolved. I could change the start and target nodes based in the (index) values generated from my map and it worked!

Now I need your suggestions if I want to get the fastest route (based on travel time not distance). What should I do? should I add these data into the map osm file? so I have to search for such info/tags. or should this be modified into the algorithms weights? I see it's mainly based on the distance. I'm really confused what to do next!

05 Oct 2013 Reham

Hello Ioannis,

I'm sorry for my previous comment, I wasn't setting the path correctly inside Matlab. Now the file runs but I have this:
Warning: path is empty. This means that the planner found no path.
> In plot_route at 24
In my_example at 37

the ways are plotted but the shortest route is not! Should I change the start and target nodes values?

and based on what? should I copy/paste nodes IDs from the XML file? knowing that I've already tried this but I get an error that the node ID is invalid.

Please I need your help since I have to modify all of the code to work on fastest route instead of the shortest as a part of my thesis, and I need to resolve all these errors and warnings first and start thinking on modifying the algorithm and add traffic or travel time info and start working on the map and algorithm to produce the fastest route. I still have no clue how to implement this, since I'm totally new to Matlab. But if you could at least help with the file you provided I'd be grateful.

Thanks,

Best,

28 Sep 2013 Reham

Hello,
I'd like to thank you for this submission, it's really helpful in my masters thesis, but I really need your help in this error I get when I run the usage_example:

Reference to non-existent field 'osm'.

Error in parse_openstreetmap (line 42)
osm_xml = map_osm.osm;

Error in usage_example (line 29)
[parsed_osm, osm_xml] = parse_openstreetmap(openstreetmap_filename);

knowing that I copied the map.osm file in the home directory of openstreetmap and I've traced the code line by line and referred to each function to see how it works and try to solve the problem but nothing, and found no similar errors in the comments below. I can't even see the output of your code in the end. So could you please help?
Thanks in advance

23 Sep 2013 Matthew

I wanted to give this 5 stars. May I suggest the following issues are addressed:
1. The usage_example throws an error: there is no restorehold() function within the distribution.
2. The chosen XML parser is not appropriate for anything but the extremely simple example.
3. The example does not work out of the box. Why not included a example map.osm file?

27 Jul 2013 Ioannis Filippidis

Hi Nermeen,

Thank you, I would suggest against parsing 30GB files in any context, MATLAB certainly cannot handle that, but neither does any other tool. The issue is that your RAM and virtual memory won't support this, unless you have a very unusual hardware or software setup.

So what I would suggest is file-based operations, where you gradually process your input, while at the same time dumping the results into a (possibly big) output file. Then you can access them from that file.

You may also consider if your task really requires brute-force parsing of whole countries, vs somehow pre-filtering the information at the source from where you get it (I don't know if that's possible in this case, but it's an option worth considering, if you don't need all that data).

The PBF format is currently not supported by this toolbox. There is already a Python solution here:
http://blog.lifeeth.in/2011/02/extract-pois-from-osm-pbf.html
(pointed to by this: http://wiki.openstreetmap.org/wiki/PBF_Format)
and I'm suggesting switching to Python anyway (see below).

Please feel free to contribute towards that, btw motivated by people's interest, it will probably migrate to github not far in the future.

Finally I would advise switching to Python for heavy GIS tasks as the one you describe, as it is more suited for system-oriented operations and you will find a much wider set of tools already available, though it takes more searching. Starting points (though haven't tried them):
http://wiki.openstreetmap.org/wiki/PythonOsmApi
http://stackoverflow.com/questions/11321718/how-can-i-display-osm-tiles-using-python/11325174#11325174
https://ep2013.europython.eu/media/conference/slides/making-use-of-openstreetmap-data-with-python.pdf
http://nzjrs.github.io/osm-gps-map/

23 Jul 2013 Nermeen Ahmed

Hi,

The tool has been magically helpful in my environment classification project. I was just wondering about the osm file size limitation that can be parsed. As i need to parse files of complete countries; osm files that can reach up to 30 GB. Also is there a way to parse PBF files instead of XML using this tool? Thanks in advance

22 Jul 2013 Ioannis Filippidis

Happy that it proves useful. No, I haven't considered processing administrative boundaries. If you write some enhancement, please consider sending it in.

Also, when I find some time this project will probably appear in github for people to contribute.

21 Jul 2013 Seb Biass

Hi there,

This toolbox is brilliant. I was wondering if you saw a way to extract and perform connectivity analysis on a road network that is within an administrative boundary of a given level. I am looking at the API, but fairly new, so still struggling...
Please do let me know if you do, and again, a bunch of thanks!

02 Jun 2013 Preeti Goel

I get an error when I try to load and parse an osm file, I have tried with different file sizes:

Cell contents reference from a non-cell array object.

Error in parse_osm>parse_way (line 61)
ndtemp(1, j) = str2double(waytemp.nd{j}.Attributes.ref);

Error in parse_osm (line 20)
parsed_osm.way = parse_way(osm_xml.way);

Error in parse_openstreetmap (line 43)
parsed_osm = parse_osm(osm_xml);

Error in OSMExample (line 29)
[parsed_osm, osm_xml] = parse_openstreetmap(openstreetmap_filename);

Any suggestions?

31 May 2013 Rinat

I have error ??? Reference to non-existent field 'osm'. osm_xml = map_osm.osm;?

matlab file xml2struct.m
tree =
[#document: null]
theStruct =
Name: 'osm'
Attributes: [1x5 struct]
Data: ''
Children: [1x439 struct]

parse_openstreetmap.m
map_osm =
Name: 'osm'
Attributes: [1x5 struct]
Data: ''
Children: [1x439 struct]

name file
openstreetmap_filename = 'map.osm';

I do not understand where the error, using Matlab 2013a and 2009a?

11 May 2013 Ioannis Filippidis

Hi Richard,

It appears that the Java Virtual Machine run out of memory, you can increase that (depending on the limits of your system). Please see here:
http://www.mathworks.com/support/solutions/en/data/1-18I2C/
This can also be the case when plotting a lot of data.

I would suggest trying to instead reduce the piece of the map you attempt to import, based on what is the purpose for further processing.
Otherwise even if it does load after increasing the java heap size, it will still be cumbersome to process.

Another suggestion might be to break into pieces and load those separately, one at a time, extracting the structure of interest and then identifying common nodes between different patches. However this is much more involved, because it involves the global (unique) IDs of the nodes, representing them in the openstreetmap database (and file).

10 May 2013 Richard

I'm trying to get a street map of Beijing, but encountering an out of memory error:

??? Java exception occurred:
java.lang.OutOfMemoryError: Java heap space
.

Error in ==> xmlread at 91
parseResult = p.parse(fileName);

Error in ==> xml2struct at 40
xDoc = xmlread(file);

Error in ==> load_osm_xml at 26
map_osm = xml2struct(filename); % downloaded osm file

Error in ==> parse_openstreetmap at 41
map_osm = load_osm_xml(openstreetmap_filename);

Error in ==> GeoLife_main at 21
[parsed_osm, osm_xml] = parse_openstreetmap('beijing.osm');

Beijing.osm is 75.35 MB

Any suggestions?

15 Feb 2013 Ioannis Filippidis

Some recent feedback I received suggests that the xml loading error might be caused by a name conflict between the function xml2struct included with the openstreetmap functions and a function with the same name in the MATLAB Bioinformatics toolbox.

My version of the bioinformatics toolbox does not have such a function, however it does have some functions for xml handling, so probably there existed xml2struct function in previous versions.

The Bioinformatics toolbox function (when it exists) can shadow the one included with this software package, depending on their relative placement in the MATLAB path.

Please check if this is the case for you by typing:
which xml2struct
This issue might be fixed in some future version by just renaming xml2struct, although in general it is preferable to maintain original names for files from the file exchange to keep better track of code and avoid duplicates.

23 Jan 2013 Ioannis Filippidis

Hi Fabio,

Are you sure that xml2struct is in your path? It appears that the file is not loaded correctly, maybe you could check to see the fields of the structure returned, containing the loaded xml.

A minimal example reproducing the issue would help resolve it.

Best,
Ioannis

08 Nov 2012 Fabio Lopez

Hi Loannis

I have just begun to use the functions package, and as a start a wanted to run the usage example. The following error arises indistinctly of the geographical location of the file or its size:

??? Reference to non-existent field 'osm'.

Error in ==> parse_openstreetmap at 42
osm_xml = map_osm.osm;

Error in ==> usage_example at 29
[parsed_osm, osm_xml] = parse_openstreetmap(openstreetmap_filename);
Could you maybe know the reason for this error?

06 Sep 2012 Paula

Hey Ioannis!
Thanks a lot for your help! Now I’m still working at the details to develop an addition for the run of the power network. I want to find out this run for whole Germany. I got the data directly from the openstreetmap homepage, where I exported them. But I just can get a really small part of Germany, if I export the data from this homepage. Do you know another page, where I can get the whole data base of Germany in once? It would be another great help for me!
Thank you again, Paula

10 Aug 2012 Ioannis Filippidis

Hi Paula,

A connectivity matrix represents the existence of connections between nodes.
In one-way roads the connections are directed, so the connectivity matrix is not symmetric.
In other words, you can go from node a to node b, but not the opposite way.
The route planner just searches within the graph represented by the connectivity matrix, in order to find a path from the initial to the final node.
Sometimes, certain roads are assumed symmetric when extracting the connectivity, but actually they are not.
For this reason, assuming that directions do not matter may work better if the first attempt fails.
This requires that the connectivity matrix be made symmetric.
Both of these two approaches are described in lines 35-48 of the usage_example file.

Since the road network was of interest in this package (at least until now), only roads qualified as valid connections.
So the code searches for road connections only.

Roads can be identified by their special tag, which has a key and a value, see
http://wiki.openstreetmap.org/wiki/Key:highway
http://wiki.openstreetmap.org/wiki/Highway_tag_usage
http://wiki.openstreetmap.org/wiki/Highways
for the related documentation on highways.
If the object has a tag, then the function get_way_tag_key gets the key and its value.

The tag's key is called "highway" and can take on many values (those of the possible values which were of interest are defined in the "road_vals" cell array in line 39 of extract_connectivity).

However, objects which are not highways do not have a tag with key named "highway".
This is checked in line 54 of extract_connectivity.
In this case, they are ignored, as far as connectivity is concerned.

Further, even if they do have the "highway" key, they may not be of interest (i.e., have a value included in "road_vals").
In this case, they are again ignored, see line 59 of extract_connectivity.

So the first warning means that the object is not a highway.
Taking the previous into consideration, it follows that this object does not contribute to the road connectivity (since it is not a road).

This is probably the reason that no route has been found.
The connectivity matrix produced from the current version of this code does not incorporate any information about the power lines or cables.

The second warning is just a suggestion by the route planner to use a symmetric network assumption, for the reasons described previously. (i.e., in case the one-way assumption does not hold exactly. For example, this could happen when we are interested in walking only and not cars. One would need to change "road_vals" as well in this case.)

To work with power networks, you would need to extract their connectivity from the loaded openstreetmap xml data.
You can do this by altering the function extract_connectivity to identify key="power" and the appropriate values, please see
http://wiki.openstreetmap.org/wiki/WikiProject_Power_networks

Extracting various networks is a good idea, I may add this capability when I have time.
If you develop an addition towards that direction, I would be happy to add it to the current distribution.

Best regards,
Ioannis

10 Aug 2012 Paula

Hey Ioannis,
I want to create a connectivity matrix for th run of cables and powerlines. Right now I'm trying to understand the function of the connectivity matrix and the rout planner, but I'm not even able to create one simple route. It always says:
"Warning: Way has NO tag.
> In get_way_tag_key at 20
In extract_connectivity at 53"
and:
"Warning: No route found by the route planner. Try without road directions."

What does it mean exactly and what can I do now?

Thanks a lot!

31 Jul 2012 Catherine Holloway  
06 Jul 2012 Jingjing Wang

One more comments.
For the route planner, it differentiates one-way roads from normal roads only by not making the directed graph symmetric. However, it does not take into account the tag "oneway=yes" which can be extracted from the original osm file. I think maybe using that info. will make more sense.

Thanks,
Jingjing

06 Jul 2012 Jingjing Wang

Hi Loannis,

I'm using your functions to get the connectivity matrix of Shanghai road network. It seems that there are several things to be modified.
1. the tag "highway" may not be always the first tag in each <way> element so in the function 'get_way_tag_key', it should traverse all the tags to see whether there is a "highway" tag.
2. In terms of the connectivity matrix, it seems to me that you are trying to get the directed graph only containing the intersection nodes. If so, 'node1_index' should not be initiated to be the first node but an empty set '[]' since the first node not necessarily belongs to the intersection nodes.

Could you help me check it?

Thanks,
Jingjing

29 May 2012 Mate  
03 May 2012 Ioannis Filippidis

Hi Catherine,

Thank you for providing feedback on these functions. I have released an updated version which fixes this bug and adds more functionality.

Best regards,
Ioannis

17 Apr 2012 Catherine Holloway

Hi Ioannis,

Thanks for taking the time to post this. I am trying to use your functions to plot the area of Genoa, Italy but I am getting the following errors:

??? Cell contents reference from a non-cell array object.

Error in ==> get_way_tag_key at 16
key = tag{1}.Attributes.k;

Error in ==> plot_way>show_ways at 43
[key, val] = get_way_tag_key(way.tag{1,i} );

Error in ==> plot_way at 34
show_ways(ax, bounds, node, way, map_img_filename);

Error in ==> genoa at 34
plot_way(ax, parsed_osm)

Do you have any suggestions?

Thanks,
Catherine

14 Apr 2012 Ioannis Filippidis

continued:

Please consider replacing:

parse_osm>parse_way line 59
tag{1,i} = waytemp.tag;

with:

if isfield(waytemp, 'tag')
tag{1, i} = waytemp.tag;
else
tag{1, i} = [];
end

I am going to post an update, fixing this bug. It would be helpful if you can post the geographical region you exported from OpenStreetmap which caused this issue. (Particularly if you can e-mail me the .osm file).

14 Apr 2012 Ioannis Filippidis

Hi David,

Thanks for noting this behavior.
Please consider replacing:
parse_osm>parse_way, line 59

if isfield(waytemp, 'tag')
tag{1, i} = waytemp.tag;
else
tag{1, i} = [];
end

11 Apr 2012 RED STONE  
27 Mar 2012 David

Hi Ioannis

I like your OpenStreetMap Matlab functions a lot. However, for some osm files I get the following error:

[parsed_osm] = parse_openstreetmap('openstreetmap/map.osm');
??? Reference to non-existent field 'tag'.

Error in ==> parse_osm>parse_way at 59
tag{1,i} = waytemp.tag;

Error in ==> parse_osm at 14
parsed_osm.way = parse_way(osm.way);

Error in ==> parse_openstreetmap at 40
parsed_osm = parse_osm(map_osm.osm);

Do you have an idea how this can be resolved?

Regards,
David

Updates
03 May 2012

Bug fixes associated with some ways having no tags.
Included routing functionality.
Added more plotting functions.
Updated manual.

19 Sep 2013

development on github

25 Nov 2013

included restorehold.m, fix warning, rename xml2struct to avoid name shadowing

22 Aug 2014

link to github using matlabcentral itself

Contact us