Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Schema detection with leading Zeros #1454

Open
mayermart opened this issue Sep 27, 2024 · 2 comments · Fixed by OSGeo/gdal#10902
Open

Schema detection with leading Zeros #1454

mayermart opened this issue Sep 27, 2024 · 2 comments · Fixed by OSGeo/gdal#10902
Labels

Comments

@mayermart
Copy link

Hello,

When reading data from a WFS (GML), I would expect the schema to return "str:2" for a field containing "01", but it actually returns "int32". Also the value returned by the feature is "1"
Is there an option to suppress the automatic schema detection and use my own schema instead and would it affect the data?

From what I understand in the documentation, a schema is only applied in 'w' mode.

Example:

import fiona

gml = """<?xml version="1.0" encoding="utf-8"?>
<wfs:FeatureCollection timeStamp="2024-09-26T13:45:45.723+02:00" numberReturned="1" numberMatched="unknown" xmlns:gml="http://www.opengis.net/gml/3.2" xmlns:wfs="http://www.opengis.net/wfs/2.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/wfs/2.0 http://schemas.opengis.net/wfs/2.0/wfs.xsd http://www.opengis.net/gml/3.2 http://schemas.opengis.net/gml/3.2.1/gml.xsd http://www.isotc211.org/2005/gco http://schemas.opengis.net/iso/19139/20070417/gco/gco.xsd http://www.isotc211.org/2005/gmd http://schemas.opengis.net/iso/19139/20070417/gmd/gmd.xsd">
	<wfs:member>
		<SomeType gml:id="a">
			<someValue>01</someValue>
			<geometry>
				<gml:Point gml:id="b" srsName="urn:ogc:def:crs:EPSG::25832" srsDimension="2">
					<gml:pos>691412 5334916</gml:pos>
				</gml:Point>
			</geometry>
		</SomeType>
	</wfs:member>
</wfs:FeatureCollection>"""

with fiona.open(fiona.io.MemoryFile(gml.encode('utf-8')), driver='GML') as source:
    print(source.schema)
    # {'properties': {'gml_id': 'str', 'someValue': 'int32'}, 'geometry': 'Unknown'}
    for feature in source:
        print(feature.properties['someValue'])
        # 1
@sgillies
Copy link
Member

@mayermart thanks for the report and the exemplary reproducer! This is a problem upstream in GDAL. ogrinfo reports the same field type:

root@195fac25be54:/# ogrinfo /data/test.gml  SomeType -so
INFO: Open of `/data/test.gml'
      using driver `GML' successful.

Layer name: SomeType
Geometry: Unknown (any)
Feature Count: 1
Extent: (691412.000000, 5334916.000000) - (691412.000000, 5334916.000000)
Layer SRS WKT:
PROJCRS["ETRS89 / UTM zone 32N",
    BASEGEOGCRS["ETRS89",
        ENSEMBLE["European Terrestrial Reference System 1989 ensemble",
            MEMBER["European Terrestrial Reference Frame 1989"],
            MEMBER["European Terrestrial Reference Frame 1990"],
            MEMBER["European Terrestrial Reference Frame 1991"],
            MEMBER["European Terrestrial Reference Frame 1992"],
            MEMBER["European Terrestrial Reference Frame 1993"],
            MEMBER["European Terrestrial Reference Frame 1994"],
            MEMBER["European Terrestrial Reference Frame 1996"],
            MEMBER["European Terrestrial Reference Frame 1997"],
            MEMBER["European Terrestrial Reference Frame 2000"],
            MEMBER["European Terrestrial Reference Frame 2005"],
            MEMBER["European Terrestrial Reference Frame 2014"],
            MEMBER["European Terrestrial Reference Frame 2020"],
            ELLIPSOID["GRS 1980",6378137,298.257222101,
                LENGTHUNIT["metre",1]],
            ENSEMBLEACCURACY[0.1]],
        PRIMEM["Greenwich",0,
            ANGLEUNIT["degree",0.0174532925199433]],
        ID["EPSG",4258]],
    CONVERSION["UTM zone 32N",
        METHOD["Transverse Mercator",
            ID["EPSG",9807]],
        PARAMETER["Latitude of natural origin",0,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8801]],
        PARAMETER["Longitude of natural origin",9,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8802]],
        PARAMETER["Scale factor at natural origin",0.9996,
            SCALEUNIT["unity",1],
            ID["EPSG",8805]],
        PARAMETER["False easting",500000,
            LENGTHUNIT["metre",1],
            ID["EPSG",8806]],
        PARAMETER["False northing",0,
            LENGTHUNIT["metre",1],
            ID["EPSG",8807]]],
    CS[Cartesian,2],
        AXIS["(E)",east,
            ORDER[1],
            LENGTHUNIT["metre",1]],
        AXIS["(N)",north,
            ORDER[2],
            LENGTHUNIT["metre",1]],
    USAGE[
        SCOPE["Engineering survey, topographic mapping."],
        AREA["Europe between 6°E and 12°E: Austria; Belgium; Denmark - onshore and offshore; Germany - onshore and offshore; Norway including - onshore and offshore; Spain - offshore."],
        BBOX[38.76,6,84.33,12.01]],
    ID["EPSG",25832]]
Data axis to CRS axis mapping: 1,2
Geometry Column = geometry
gml_id: String (0.0) NOT NULL
someValue: Integer (0.0)

If the OGR GML driver doesn't provide some specific options to manage this, can you make a bug report in the GDAL repo and link here?

@mayermart
Copy link
Author

Sorry, I was too busy to open a separate issue on the GDAL repository.
However, it seems to have been resolved by OSGeo/gdal#10902.
Although it may not yet be available through Fiona, should we close this issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants