Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unable to generate object with minProperties or set minimum required properties. #113

Open
danmash opened this issue May 10, 2024 · 14 comments · May be fixed by #114
Open

Unable to generate object with minProperties or set minimum required properties. #113

danmash opened this issue May 10, 2024 · 14 comments · May be fixed by #114

Comments

@danmash
Copy link

danmash commented May 10, 2024 •
edited
Loading

Hi, I'm trying to generate the following dict containing up to 3 items with True values:

{'new_parent': True, 'new_subsidiary': True, 'new_sibling': True}

However, I don't see a correct JSON schema to make this done with JSF. When I try to use anyOf I could get False. On the other hand, if I'm trying to use enum JSF generates 1.

jsf==0.8.0, (with jsf==0.11.2 I get slightly different results, see comment)

In [1]: parameters_schema = {
   ...:         "type": "object",
   ...:         "friendly_name": "Corporate Structure Changes",
   ...:         "description": "Parameters for corporate structure changes",
   ...:         "properties": {
   ...:             "new_parent": {"type": "boolean"},
   ...:             "new_subsidiary": {"type": "boolean"},
   ...:             "new_sibling": {"type": "boolean"},
   ...:         },
   ...:         "anyOf": [
   ...:             {"properties": {"new_parent": {"enum": [True]}}},
   ...:             {"properties": {"new_subsidiary": {"enum": [True]}}},
   ...:             {"properties": {"new_sibling": {"enum": [True]}}},
   ...:         ]
   ...:     }

In [2]: from jsf import JSF
   ...: 

In [3]: JSF(parameters_schema).generate()
Out[3]: {}

In [4]: JSF(parameters_schema).generate()
Out[4]: {'new_parent': True, 'new_sibling': True}

In [5]: JSF(parameters_schema).generate()
Out[5]: {'new_subsidiary': True, 'new_sibling': False}

In [6]: JSF(parameters_schema).generate()
Out[6]: {'new_parent': False, 'new_subsidiary': False}

In [7]: JSF(parameters_schema).generate()
Out[7]: {'new_parent': True}

In [8]: parameters_schema = {
   ...:     "type": "object",
   ...:     "friendly_name": "Corporate Structure Changes",
   ...:     "description": "Parameters for corporate structure changes",
   ...:     "properties": {
   ...:         "new_parent": {"type": "boolean", "enum": [True]},
   ...:         "new_subsidiary": {"type": "boolean", "enum": [True]},
   ...:         "new_sibling": {"type": "boolean", "enum": [True]},
   ...:     },
   ...: }

In [9]: JSF(parameters_schema).generate()
Out[9]: {'new_subsidiary': 1, 'new_sibling': 1}

In [10]: JSF(parameters_schema).generate()
Out[10]: {'new_sibling': 1}

In [11]:     parameters_schema = {
    ...:         "type": "object",
    ...:         "friendly_name": "Corporate Structure Changes",
    ...:         "description": "Parameters for corporate structure changes",
    ...:         "properties": {
    ...:             "new_parent": {"type": "boolean", "anyOf": [{"enum": [True]}]},
    ...:             "new_subsidiary": {"type": "boolean", "anyOf": [{"enum": [True]}]},
    ...:             "new_sibling": {"type": "boolean",  "anyOf": [{"enum": [True]}]},
    ...:         }
    ...:     }

In [12]: JSF(parameters_schema).generate()
Out[12]: {'new_parent': False, 'new_subsidiary': True}

In [13]: JSF(parameters_schema).generate()
Out[13]: {'new_parent': True, 'new_sibling': True}

In [14]: JSF(parameters_schema).generate()
Out[14]: {'new_sibling': True}

In [15]: JSF(parameters_schema).generate()
Out[15]: {'new_parent': False, 'new_subsidiary': True}

In [16]:     parameters_schema = {
    ...:         "type": "object",
    ...:         "friendly_name": "Corporate Structure Changes",
    ...:         "description": "Parameters for corporate structure changes",
    ...:         "properties": {
    ...:             "new_parent": {"type": "boolean", "enum": [True]},
    ...:             "new_subsidiary": {"type": "boolean", "enum": [True]},
    ...:             "new_sibling": {"type": "boolean", "enum": [True]},
    ...:         }
    ...:     }

In [17]: JSF(parameters_schema).generate()
Out[17]: {'new_parent': 1, 'new_subsidiary': 1, 'new_sibling': 1}

In [18]:     parameters_schema = {
    ...:         "type": "object",
    ...:         "friendly_name": "Corporate Structure Changes",
    ...:         "description": "Parameters for corporate structure changes",
    ...:         "properties": {
    ...:             "new_parent": {"type": "boolean", "allOf": [{"enum": [True]}]},
    ...:             "new_subsidiary": {"type": "boolean", "allOf": [{"enum": [True]}]},
    ...:             "new_sibling": {"type": "boolean",  "allOf": [{"enum": [True]}]},
    ...:         }
    ...:     }

In [19]: JSF(parameters_schema).generate()
Out[19]: {'new_subsidiary': False, 'new_sibling': True}

In [20]: JSF(parameters_schema).generate()
Out[20]: {'new_subsidiary': False, 'new_sibling': True}

In [21]: JSF(parameters_schema).generate()
Out[21]: {'new_sibling': False}

In [22]:     parameters_schema = {
    ...:         "type": "object",
    ...:         "friendly_name": "Corporate Structure Changes",
    ...:         "description": "Parameters for corporate structure changes",
    ...:         "properties": {
    ...:             "new_parent": {"anyOf": [{"enum": [True], "type": "boolean"}]},
    ...:             "new_sibling": {"anyOf": [{"enum": [True], "type": "boolean"}]},
    ...:             "new_subsidiary": {"anyOf": [{"enum": [True], "type": "boolean"}]},
    ...:         }
    ...:     }

In [23]: JSF(parameters_schema).generate()
Out[23]: {'new_parent': 1, 'new_sibling': 1}
@danmash
Copy link
Author

danmash commented May 10, 2024 •
edited
Loading

jsf==0.11.2 
In [1]:     parameters_schema = {
   ...:         "type": "object",
   ...:         "friendly_name": "Corporate Structure Changes",
   ...:         "description": "Parameters for corporate structure changes",
   ...:         "properties": {
   ...:             "new_parent": {"type": "boolean", "enum": [True]},
   ...:             "new_subsidiary": {"type": "boolean", "enum": [True]},
   ...:             "new_sibling": {"type": "boolean", "enum": [True]},
   ...:         }
   ...:     }

In [2]: from jsf import JSF

In [3]: JSF(parameters_schema).generate()
Out[3]: {'new_parent': 1}

In [4]:     parameters_schema = {
   ...:         "type": "object",
   ...:         "friendly_name": "Corporate Structure Changes",
   ...:         "description": "Parameters for corporate structure changes",
   ...:         "properties": {
   ...:             "new_parent": {"anyOf": [{"enum": [True], "type": "boolean"}]},
   ...:             "new_sibling": {"anyOf": [{"enum": [True], "type": "boolean"}]},
   ...:             "new_subsidiary": {"anyOf": [{"enum": [True], "type": "boolean"}]},
   ...:         }
   ...:     }

In [5]: JSF(parameters_schema).generate()
Out[5]: {'new_sibling': 1}

In [6]:     parameters_schema = {
   ...:         "type": "object",
   ...:         "friendly_name": "Corporate Structure Changes",
   ...:         "description": "Parameters for corporate structure changes",
   ...:         "properties": {
   ...:             "new_parent": {"type": "boolean", "allOf": [{"enum": [True]}]},
   ...:             "new_subsidiary": {"type": "boolean", "allOf": [{"enum": [True]}]},
   ...:             "new_sibling": {"type": "boolean",  "allOf": [{"enum": [True]}]},
   ...:         }
   ...:     }

In [7]: JSF(parameters_schema).generate()
Out[7]: {}

In [8]: JSF(parameters_schema).generate()
Out[8]: {'new_subsidiary': False}

In [9]: JSF(parameters_schema).generate()
Out[9]: {'new_parent': True, 'new_sibling': False}

In [10]:     parameters_schema = {
    ...:         "type": "object",
    ...:         "friendly_name": "Corporate Structure Changes",
    ...:         "description": "Parameters for corporate structure changes",
    ...:         "properties": {
    ...:             "new_parent": {"type": "boolean"},
    ...:             "new_subsidiary": {"type": "boolean"},
    ...:             "new_sibling": {"type": "boolean"},
    ...:         },
    ...:         "anyOf": [
    ...:             {"properties": {"new_parent": {"const": True}}},
    ...:             {"properties": {"new_subsidiary": {"const": True}}},
    ...:             {"properties": {"new_sibling": {"const": True}}},
    ...:         ]
    ...:     }

In [11]: JSF(parameters_schema).generate()
Out[11]: {'new_subsidiary': False, 'new_sibling': True}

In [12]: JSF(parameters_schema).generate()
Out[12]: {'new_parent': False, 'new_sibling': True}

In [13]:     parameters_schema = {
    ...:         "type": "object",
    ...:         "friendly_name": "Corporate Structure Changes",
    ...:         "description": "Parameters for corporate structure changes",
    ...:         "properties": {
    ...:             "new_parent": {"type": "boolean", "const": True},
    ...:             "new_subsidiary": {"type": "boolean", "const": True},
    ...:             "new_sibling": {"type": "boolean", "const": True},
    ...:         },
    ...:     }

In [14]: JSF(parameters_schema).generate()
Out[14]: {}

In [15]: JSF(parameters_schema).generate()
Out[15]: {'new_subsidiary': 1}

In [16]: JSF(parameters_schema).generate()
Out[16]: {'new_subsidiary': 1}

@ghandic
Copy link
Owner

ghandic commented May 10, 2024

If you want only true but it's still a bool in the schema you can use default or const not enum

@danmash
Copy link
Author

danmash commented May 13, 2024

@ghandic none of these seems to work

In [20]:     parameters_schema = {
    ...:         "type": "object",
    ...:         "friendly_name": "Corporate Structure Changes",
    ...:         "description": "Parameters for corporate structure changes",
    ...:         "properties": {
    ...:             "new_parent": {"type": "boolean", "default": True},
    ...:             "new_subsidiary": {"type": "boolean", "default": True},
    ...:             "new_sibling": {"type": "boolean", "default": True},
    ...:         },
    ...:     }

In [21]: JSF(parameters_schema).generate()
Out[21]: {}

In [22]: JSF(parameters_schema).generate()
Out[22]: {'new_subsidiary': False}

In [23]: JSF(parameters_schema).generate()
Out[23]: {'new_subsidiary': True, 'new_sibling': False}

In [24]:     parameters_schema = {
    ...:         "type": "object",
    ...:         "friendly_name": "Corporate Structure Changes",
    ...:         "description": "Parameters for corporate structure changes",
    ...:         "properties": {
    ...:             "new_parent": {"type": "boolean", "const": True},
    ...:             "new_subsidiary": {"type": "boolean", "const": True},
    ...:             "new_sibling": {"type": "boolean", "const": True},
    ...:         },
    ...:     }

In [25]: JSF(parameters_schema).generate()
Out[25]: {'new_subsidiary': 1, 'new_sibling': 1}

@ghandic
Copy link
Owner

ghandic commented May 13, 2024

Could you try adding required?

@danmash
Copy link
Author

danmash commented May 13, 2024

@ghandic these properties should not be required, my goal is to use "anyOf" to make sure at least one of the properties exists with "True" value.

 In [26]:     parameters_schema = {
    ...:         "type": "object",
    ...:         "friendly_name": "Corporate Structure Changes",
    ...:         "description": "Parameters for corporate structure changes",
    ...:         "properties": {
    ...:             "new_parent": {"type": "boolean"},
    ...:             "new_subsidiary": {"type": "boolean"},
    ...:             "new_sibling": {"type": "boolean"},
    ...:         },
    ...:         "anyOf": [
    ...:             {"properties": {"new_parent": {"const": True}}},
    ...:             {"properties": {"new_subsidiary": {"const": True}}},
    ...:             {"properties": {"new_sibling": {"const": True}}},
    ...:         ]
    ...:     }

In [27]: JSF(parameters_schema).generate()
Out[27]: {'new_subsidiary': True}

In [28]: JSF(parameters_schema).generate()
Out[28]: {'new_subsidiary': False}

@ghandic
Copy link
Owner

ghandic commented May 13, 2024

I think what you're wanting would be

from jsf import JSF

parameters_schema = {
    "type": "object",
    "friendly_name": "Corporate Structure Changes",
    "description": "Parameters for corporate structure changes",
    "properties": {
        "new_parent": {"type": "boolean", "default": True},
        "new_subsidiary": {"type": "boolean", "default": True},
        "new_sibling": {"type": "boolean", "default": True},
    },
    "minProperties": 1,
}

for _ in range(5):
    print(JSF(parameters_schema).generate(use_defaults=True))

But we don't currently support minProperties, but this ensures only true values are set but not that at least one is set.

@ghandic ghandic changed the title Unable to generate data with True values only. Unable to generate object with minProperties or set minimum required properties. May 13, 2024
@danmash
Copy link
Author

danmash commented May 13, 2024 •
edited
Loading

@ghandic Thank you for the proposed solution which is actually generated that I need. The only problem with using default I see that it removes the opposite operation - validation of JSON based on the JSON schema.

In our project we use JSON schema for validation and I use JSF in our unit tests to randomly generate data for tests.
It's true that when I switch from "enum" to "default" I'm able to generate the JSON needed, but I'm loosing the validation aspect of the JSON schema.

Original JSON schema we use for validation:

    parameters_schema = {
        "type": "object",
        "friendly_name": "Corporate Structure Changes",
        "description": "Parameters for corporate structure changes",
        "properties": {
            "new_parent": {"type": "boolean"},
            "new_subsidiary": {"type": "boolean"},
            "new_sibling": {"type": "boolean"},
        },
        "anyOf": [
            {"properties": {"new_parent": {"enum": [True]}}},
            {"properties": {"new_subsidiary": {"enum": [True]}}},
            {"properties": {"new_sibling": {"enum": [True]}}},
        ]
    }

Sorry for the misleading issue description. I hope now my task is clear. Thank you!

@ghandic
Copy link
Owner

ghandic commented May 13, 2024 •
edited
Loading

You can use examples

from jsf import JSF

parameters_schema = {
    "type": "object",
    "friendly_name": "Corporate Structure Changes",
    "description": "Parameters for corporate structure changes",
    "properties": {
        "new_parent": {"type": "boolean", "examples": [True]},
        "new_subsidiary": {"type": "boolean", "examples": [True]},
        "new_sibling": {"type": "boolean", "examples": [True]},
    },
}

for _ in range(5):
    print(JSF(parameters_schema).generate(use_examples=True))

@danmash
Copy link
Author

danmash commented May 13, 2024 •
edited
Loading

@ghandic examples still doesn't validate data like constant + minProperties does

In [42]: parameters_schema = {
    ...:     "type": "object",
    ...:     "friendly_name": "Corporate Structure Changes",
    ...:     "description": "Parameters for corporate structure changes",
    ...:     "properties": {
    ...:         "new_parent": {"type": "boolean", "examples": [True]},
    ...:         "new_subsidiary": {"type": "boolean", "examples": [True]},
    ...:         "new_sibling": {"type": "boolean", "examples": [True]},
    ...:     },
    ...: }

In [43]: for _ in range(5):
    ...:     print(JSF(parameters_schema).generate(use_examples=True))
    ...: 
{'new_parent': True, 'new_subsidiary': True}
{'new_subsidiary': True}
{'new_parent': True, 'new_subsidiary': True}
{'new_parent': True, 'new_subsidiary': True}
{'new_subsidiary': True}

In [44]: import jsonschema

In [45]: jsonschema.validate({'new_parent': False, 'new_sibling': False}, parameters_schema)

In [47]:

In [56]: parameters_schema = {
    ...:     "type": "object",
    ...:     "friendly_name": "Corporate Structure Changes",
    ...:     "description": "Parameters for corporate structure changes",
    ...:     "properties": {
    ...:         "new_parent": {"type": "boolean", "const": True},
    ...:         "new_subsidiary": {"type": "boolean", "const": True},
    ...:         "new_sibling": {"type": "boolean", "const": True},
    ...:     },
    ...:     "minProperties": 1,
    ...: }

In [57]: jsonschema.validate({'new_parent': False, 'new_sibling': False}, parameters_schema)
---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
Cell In[57], line 1
----> 1 jsonschema.validate({'new_parent': False, 'new_sibling': False}, parameters_schema)

File ~/.pyenv/versions/3.8.17/envs/zint-django/lib/python3.8/site-packages/jsonschema/validators.py:1332, in validate(instance, schema, cls, *args, **kwargs)
   1330 error = exceptions.best_match(validator.iter_errors(instance))
   1331 if error is not None:
-> 1332     raise error

ValidationError: True was expected

Failed validating 'const' in schema['properties']['new_sibling']:
    {'const': True, 'type': 'boolean'}

On instance['new_sibling']:
    False

@ghandic
Copy link
Owner

ghandic commented May 13, 2024

Ahh so the type isnt bool, you want it True all the time and the problem being when you set "const": True it converts to an int?

@ghandic ghandic linked a pull request May 13, 2024 that will close this issue
Open
@ghandic
Copy link
Owner

ghandic commented May 13, 2024

Could you have a squiz at the PR - see if that addresses your problem

@danmash
Copy link
Author

danmash commented May 13, 2024

Hi @ghandic , sorry for the long reply. I double-check with my team about what we are trying to achieve. I added changes to your branch #115
Below is an example of data validation

schema = {
    "$schema": "http://json-schema.org/draft-07/schema#",
    "type": "object",
    "properties": {
        "new_parent": {"type": "boolean"},
        "new_subsidiary": {"type": "boolean"},
        "new_sibling": {"type": "boolean"},
    },
    "additionalProperties": False,
    
            "anyOf": [
                {"properties": {"new_parent": {"const": True}}},
                {"properties": {"new_subsidiary": {"const": True}}},
                {"properties": {"new_sibling": {"const": True}}},
            ]
        
    ,
}

valid_data1 = {"new_parent": True, "new_sibling": False}
valid_data2 = {"new_subsidiary": True, "new_parent": False, "new_sibling": True}
invalid_data = {"new_parent": False, "new_subsidiary": False, "new_sibling": False}

for data in [valid_data1, valid_data2, invalid_data]:
    try:
        jsonschema.validate(data, schema)
        print(f"{data} is valid")
    except jsonschema.exceptions.ValidationError as e:
        print(f"{data} is invalid: {e}")

as you can see third sample is invalid, but jsf still generates invalid data sometimes

In [6]: JSF(schema).generate()
Out[6]: {'new_subsidiary': False, 'new_sibling': False}

In [9]: JSF(schema).generate()
Out[9]: {}

In [12]: JSF(schema).generate()
Out[12]: {'new_sibling': False}

It would be nice to have "minProperties": 1 as well, to avoid having empty dict generation

@ghandic
Copy link
Owner

ghandic commented May 13, 2024

That's what the required does in the anyof/allof/oneof

Min properties doesn't scale well as if you add another optional property it wouldn't work

@danmash
Copy link
Author

danmash commented May 14, 2024 •
edited
Loading

@ghandic here's schema we need without minProperties and with required as you suggested
False is allowed, but at least one value should be True

    parameters_schema = {
        "type": "object",
        "friendly_name": "Corporate Structure Changes",
        "description": "Parameters for corporate structure changes",
        "properties": {
            "new_parent": {"type": "boolean"},
            "new_subsidiary": {"type": "boolean"},
            "new_sibling": {"type": "boolean"},
        },
        "anyOf": [
            {"properties": {"new_parent": {"const": True}}, "required": ["new_parent"]},
            {"properties": {"new_subsidiary": {"const": True}}, "required": ["new_subsidiary"]},
            {"properties": {"new_sibling": {"const": True}}, "required": ["new_sibling"]},
        ]
    }

and JSF generates data which cannot pass the validation.

In [65]: JSF(parameters_schema).generate()
Out[65]: {}

In [81]: JSF(parameters_schema).generate()
Out[81]: {'new_parent': False, 'new_subsidiary': False}

In [82]: jsonschema.validate({"new_parent": False, "new_subsidiary": False},parameters_schema)
---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
Cell In[82], line 1
----> 1 jsonschema.validate({"new_parent": False, "new_subsidiary": False},parameters_schema)

File ~/.pyenv/versions/3.8.17/envs/zint-django/lib/python3.8/site-packages/jsonschema/validators.py:1332, in validate(instance, schema, cls, *args, **kwargs)
   1330 error = exceptions.best_match(validator.iter_errors(instance))
   1331 if error is not None:
-> 1332     raise error

ValidationError: True was expected

Failed validating 'const' in schema[0]['properties']['new_parent']:
    {'const': True}

On instance['new_parent']:
    False

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix: Minimum required props using oneOf ghandic/jsf
2 participants
@danmash @ghandic