Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GPDSC Spec clarification]: Do "csolution" uses the GPDSC file content to override the PDSC definition of components ? #317

Open
BadreddineBJA opened this issue May 28, 2024 · 6 comments
Assignees
Labels
documentation Improvements or additions to documentation question Further information is requested

Comments

@BadreddineBJA
Copy link

BadreddineBJA commented May 28, 2024

GPDSC Spec clarification

1. Assumptions:

  • The GPDSC .xsd schema is the same as the PDSC schema, hence any PDSC element declared in the spec should be supported by the GPDSC.
  • The GPDSC is parsed by csolution and used as a mean to override the PDSC definition of components.

2. Scenario example description:

a. Use a packA PDSC definition like the following:

<?xml version="1.0" encoding="UTF-8" ?>
<package xmlns:xs="http://www.w3.org/2001/XMLSchema-instance" schemaVersion="1.7.2" xs:noNamespaceSchemaLocation="https://github.com/Open-CMSIS-Pack/Open-CMSIS-Pack-Spec/blob/main/schema/PACK.xsd">
    <vendor>STMicro</vendor>
    <name>packA</name>
    <description>Pack A description</description>
    <generators>
        <generator id="GEN-1">
            <description>GEN1 generator</description>
            <exe>
                <command host="win">gen.bat</command>
                <command host="linux">gen.sh</command>
                <!-- Generator Id -->
                <argument>GEN-1</argument>
                <!-- Generator Input File absolute path -->
                <argument>$G</argument>
                <!-- GPDSC Template File Name -->
                <argument>template_model.gpdsc.hbs</argument>
            </exe>
            <gpdsc name="$P/template_model.gpdsc"/>
        </generator>
    </generators>
    <components>
        <component Cclass="Device" Cgroup="generated" Csub="compA" Cversion="0.0.1" generator="GEN-1">
            <description>compA description</description>
            <RTE_Components_h>
                <!-- the following content goes into file 'RTE_Components.h' -->
				#define _RTE_COMPA_FROM_PDSC
            </RTE_Components_h>
            <Pre_Include_Global_h>
                <!-- the following content goes into file 'Pre_Include_Global.h' -->
				#define PRE_INCLUDE_GLOBAL_COMPA_FROM_PDSC
            </Pre_Include_Global_h>
            <Pre_Include_Local_Component_h>
                <!-- the following content goes into file 'Pre_Include_Local_<ComponentId>.h' -->
				#define PRE_INCLUDE_LOCAL_COMPA_FROM_PDSC
            </Pre_Include_Local_Component_h>
            <files>
                <file category="genAsset" name="templates/compA/stm32_compA_template.c"/>
                <file category="genAsset" name="templates/compA/stm32_compA_template.h"/>
            </files>
        </component>
    </components>
</package>

b. Generate a GPDSC file with the following content:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- ******************************************************************************
 * File Name   : GPDSC_template.gpdsc
 * Description : Generator PDSC File generated (DO NOT EDIT!)
 ****************************************************************************** -->

<package xmlns:xs="http://www.w3.org/2001/XMLSchema-instance" schemaVersion="1.0"
    xs:noNamespaceSchemaLocation="https://github.com/Open-CMSIS-Pack/Open-CMSIS-Pack-Spec/blob/main/schema/PACK.xsd">
    <vendor>STMicro</vendor>
    <name>packA</name>
    <description>Pack A description</description>
    <url>path</url>
    <generators>
        <generator id="GEN-1">
            <description>GEN1 generator</description>
            <exe>
                <command host="win">gen.bat</command>
                <command host="linux">gen.sh</command>
                <!-- Generator Id -->
                <argument>GEN-1</argument>
                <!-- Generator Input File absolute path -->
                <argument>$G</argument>
                <!-- GPDSC Template File Name -->
                <argument>template_model.gpdsc.hbs</argument>
            </exe>
            <gpdsc name="$P/template_model.gpdsc" />
        </generator>
    </generators>
    <components>
    	<component Cclass="Device" Cgroup="generated" Csub="compA" Cversion="1.0.0" generator="GEN-1">
            <description>compA description</description>
            <RTE_Components_h>
                <!-- the following content goes into file 'RTE_Components.h' -->
                #define USING_CompA_GPDSC
            </RTE_Components_h>
            <Pre_Include_Global_h>
                <!-- the following content goes into file 'Pre_Include_Global.h' -->
                #define PRE_INCLUDE_GLOBAL_compA_GPDSC
            </Pre_Include_Global_h>
            <Pre_Include_Local_Component_h>
                <!-- the following content goes into file 'Pre_Include_Local_<ComponentId>.h' -->
                #define PRE_INCLUDE_LOCAL_compA_GPDSC
            </Pre_Include_Local_Component_h>
            <files>
                <file category="source" name="stm32_compA.c" />
                <file category="header" name="stm32_compA.h" />
            </files>
        </component>
    </components>    
</package>

3. Observed Result:
The content of the RTE_<context_id> Generated files is the PDSC list of defines, the GPDSC definition of <RTE_Components_h>, <Pre_Include_Global_h>, <Pre_Include_Local_Component_h> for the same component is not taken into consideration by csolution.

RTE_Components.h generated file:

/*
 * CSOLUTION generated file: DO NOT EDIT!
 * Generated by: csolution version 2.0.0
 * Target:  'debug+main' 
 */
#ifndef RTE_COMPONENTS_H
#define RTE_COMPONENTS_H
/*
 * Define the Device Header File: 
 */
#define CMSIS_device_header "stm32u5xx.h"

/* STMicro::Device:generated:compA:0.0.1 */
#define _RTE_COMPA_FROM_PDSC**

#endif /* RTE_COMPONENTS_H */

Pre_Include_Global.h

/*
 * CSOLUTION generated file: DO NOT EDIT!
 * Generated by: csolution version 2.0.0
 * Target:  'debug+main' 
 */
#ifndef PRE_INCLUDE_GLOBAL_H
#define PRE_INCLUDE_GLOBAL_H
/* STMicro::Device:generated:compA:0.0.1 */
#define PRE_INCLUDE_GLOBAL_COMPA_FROM_PDSC**
#endif /* PRE_INCLUDE_GLOBAL_H */

Pre_Include_Local_compId.h

/*
 * CSOLUTION generated file: DO NOT EDIT!
 * Generated by: csolution version 2.0.0
 * Target:  'debug+main' 
 */

#ifndef PRE_INCLUDE_DEVICE_GENERATED_COMPA_H
#define PRE_INCLUDE_DEVICE_GENERATED_COMPA_H

/* STMicro::Device:generated:compA:0.0.1 */
#define PRE_INCLUDE_LOCAL_COMPA_FROM_PDSC

#endif /* PRE_INCLUDE_DEVICE_GENERATED_COMPA_H */

4. What is expected ?

  • GPDSC "component A" tags definitions ( <RTE_Components_h>, <Pre_Include_Global_h>, <Pre_Include_Local_Component_h> ) should override the PDSC definitions of the same component.
  • General use case: Any GPDSC component definition should override the same PDSC component definition at the convert step.
@BadreddineBJA BadreddineBJA changed the title GPDSC Spec clarification: is the GPDSC file used for PDSC elements override [GPDSC Spec clarification]: Do "csolution" uses the GPDSC file content to override the PDSC definition of components ? May 28, 2024
@BadreddineBJA
Copy link
Author

BadreddineBJA commented May 28, 2024

Tests re-executed using csolution v2.3.0, I get the same result as csolution v2.0.0, only the PDSC defines are taken into consideration, GPDSC declared defines are not taken into account at the convert step

RTE_Components.h generated file:

/*
 * CSOLUTION generated file: DO NOT EDIT!
 * Generated by: csolution version 2.3.0
 *
 * Project: 'app6_SW.debug+main' 
 * Target:  'debug+main' 
 */

#ifndef RTE_COMPONENTS_H
#define RTE_COMPONENTS_H


/*
 * Define the Device Header File: 
 */
#define CMSIS_device_header "stm32u5xx.h"

/* STMicro::Device:generated:[email protected] */
#define _RTE_COMPA_FROM_PDSC


#endif /* RTE_COMPONENTS_H */

Pre_Include_Global.h

/*
 * CSOLUTION generated file: DO NOT EDIT!
 * Generated by: csolution version 2.3.0
 *
 * Project: 'app6_SW.debug+main' 
 * Target:  'debug+main' 
 */

#ifndef PRE_INCLUDE_GLOBAL_H
#define PRE_INCLUDE_GLOBAL_H

/* STMicro::Device:generated:[email protected] */
#define PRE_INCLUDE_GLOBAL_COMPA_FROM_PDSC


#endif /* PRE_INCLUDE_GLOBAL_H */

Pre_Include_Local_compId.h

/*
 * CSOLUTION generated file: DO NOT EDIT!
 * Generated by: csolution version 2.3.0
 *
 * Project: 'app6_SW.debug+main' 
 * Target:  'debug+main' 
 */

#ifndef PRE_INCLUDE_DEVICE_GENERATED_COMPA_H
#define PRE_INCLUDE_DEVICE_GENERATED_COMPA_H

/* STMicro::Device:generated:[email protected] */
#define PRE_INCLUDE_LOCAL_COMPA_FROM_PDSC

#endif /* PRE_INCLUDE_DEVICE_GENERATED_COMPA_H */

Observed behavior:

GPDSC definition of the same component is not overriding the PDSC component element.

Extra tests:

When updating the pack vendor name of the GPDSC template, or the component cgroup like displayed in the snapshot:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- ******************************************************************************
 * File Name   : GPDSC_template.gpdsc
 * Description : Generator PDSC File generated by CodeGen (DO NOT EDIT!)
 ****************************************************************************** -->

<package xmlns:xs="http://www.w3.org/2001/XMLSchema-instance" schemaVersion="1.0"
    xs:noNamespaceSchemaLocation="https://github.com/Open-CMSIS-Pack/Open-CMSIS-Pack-Spec/blob/main/schema/PACK.xsd">
    <vendor>STMicro</vendor>
    <name>packA</name>
    <description>Pack A description</description>
    <url>path</url>
    <releases>
        <release version="0.0.1">
            {{lowercase "- First GPDSC template Model Release"}}
        </release>
    </releases>
    <conditions>
        <condition id="conditionC">
            <description>condition C</description>
            <require Cclass="fake" Cgroup="generated" Csub="compF" />
        </condition>
        <condition id="filecondition1">
            <description>Condition 1</description>
            <require Cclass="fake" Cgroup="generated" Csub="compB" />
        </condition>
        <condition id="filecondition2">
            <description>Condition 2</description>
            <require Cclass="fake" Cgroup="generated" Csub="compC" />
        </condition>
    </conditions>
    <generators>
        <generator id="GEN-1">
            <description>GEN1 generator</description>
            <exe>
                <command host="win">gen.bat</command>
                <command host="linux">gen.sh</command>
                <!-- Generator Id -->
                <argument>GEN-1</argument>
                <!-- Generator Input File absolute path -->
                <argument>$G</argument>
                <!-- GPDSC Template File Name -->
                <argument>template_model.gpdsc.hbs</argument>
                <!-- Flag to indicate that this Generator is dry-run capable  -->
                <argument mode="dry-run">--dry-run</argument>
            </exe>
            <gpdsc name="$P/template_model.gpdsc" />
        </generator>
    </generators>
    {{!-- SELECTED COMPONENTS LIST --}}

    <components>
    	<component Cclass="Device" Cgroup="generated-updated" Csub="compA" Cversion="1.0.0" generator="GEN-1">
            <description>TO BE DEFINED</description>
            <RTE_Components_h>
                <!-- the following content goes into file 'RTE_Components.h' -->
                #define USING_CompA_GPDSC
            </RTE_Components_h>
            <Pre_Include_Global_h>
                <!-- the following content goes into file 'Pre_Include_Global.h' -->
                #define PRE_INCLUDE_GLOBAL_compA_GPDSC
            </Pre_Include_Global_h>
            <Pre_Include_Local_Component_h>
                <!-- the following content goes into file 'Pre_Include_Local_<ComponentId>.h' -->
                #define PRE_INCLUDE_LOCAL_compA_GPDSC
            </Pre_Include_Local_Component_h>
            <files>
                <file category="source" name="stm32_compA.c" />
                <file category="header" name="stm32_compA.h" />
            </files>
        </component>
    </components>    
</package>

the new updated component description is taken into consideration
this is what we get in the RTE_Components.h file

/*
 * CSOLUTION generated file: DO NOT EDIT!
 * Generated by: csolution version 2.3.0
 *
 * Project: 'app6_SW.debug+main' 
 * Target:  'debug+main' 
 */

#ifndef RTE_COMPONENTS_H
#define RTE_COMPONENTS_H


/*
 * Define the Device Header File: 
 */
#define CMSIS_device_header "stm32u5xx.h"

/* STMicro::Device:generated-updated:[email protected] */
#define USING_CompA_GPDSC
/* STMicro::Device:generated:[email protected] */
#define _RTE_COMPA_FROM_PDSC


#endif /* RTE_COMPONENTS_H */

Main Question

  • For the same component Exact PDSC / GPDSC definition, what are the rules applied ?
  • Should the GPDSC override the PDSC definition ?
  • Is there any specific handling in the toolbox for the GPDSC components definition ? (Ex: components with higher versions in the GPDSC should override the PDSC ones ?)

@jkrech jkrech added the question Further information is requested label Jun 3, 2024
@jkrech jkrech added the bug Something isn't working label Jun 3, 2024
@jkrech
Copy link
Member

jkrech commented Jun 3, 2024

I reproduced a similar scenario in MDK uVision 5.40 and do see the expected behavior, that the Cversion attribute of the component definition in the GPDSC file needs to be larger or equal to the component version in the PDSC file to override it.
This includes all aspects of the component including RTE_Components.h, Global and Local Pre-Include definitions.

I converted the uvprojx to csolution and repeated the test and I can see the defines from GPDSC if the component defined in the gpdsc is greater or equal to the Cversion of the component specified in the PDSC file. Otherwise I see the defines coming from the PDSC
I used CMSIS-Toolbox 2.4.0 and 2.3.0 but was unable to reproduce the behavior you are describing above.

@jkrech jkrech assigned jkrech and unassigned brondani Jun 4, 2024
@BadreddineBJA
Copy link
Author

Thanks Joachim for the tests, I will re-check the tests following your described scenario about the component version in the GPDSC that should be greater or equal to the PDSC version and confirm the behavior.
i used the csolution "v2.3.0" from the toolbox on "windows" for previous described scenarios,

@BadreddineBJA
Copy link
Author

BadreddineBJA commented Jun 11, 2024

  • Tests re-applied using the toolbox v2.4.0.
  • Mocked packs and project example used for test: shared in this private link
  • Test results:

Following the above recommendations from @jkrech (The Cversion attribute of the component definition in the GPDSC file needs to be larger or equal to the component version in the PDSC file to override it), at the convert step, i can see in the RTE folder that the defines from GPDSC are taken into consideration if the component defined in the gpdsc is greater or equal to the Cversion of the component specified in the PDSC file. Otherwise I see the defines coming from the PDSC.

. I think that this behavior should be documented in the GPDSC spec

For the rest of the applied tests, the GPDSC template manual editing is very error prone since it can cause GPDSC .xsd schema inconsistancies which seems implicitly not detected by csolution (maybe warnings messages to the end user should be added to ease the debug step)

@BadreddineBJA
Copy link
Author

BadreddineBJA commented Jun 11, 2024

Hi @jkrech,

Thank you for your feedback
Regarding your question, Do you make use of config files in your component descriptions for the generator component and the overwrite in the generated gpdsc?

  • No, currently In the applied tests, I am not using the attr=”config” in the gpdsc template.

Following our discussion about the strange behaviors observed and described in this ticket, I re-adjusted the GPDSC template following your recommendation:

    1. I added the same node in the GPDSC template.
    1. I Checked for any open / malformed tag in the GPDSC (applied a packchk on the gpdsc .xsd schema).
    1. I verified if the GPDSC component used version is greater or equal to the PDSC component version.

After following this scenario:

  1. Compose a project using 2 components: compA, compB .
  2. Generate a GPDSC file with the same PDSC definition of this components (Cclass, Cgroup, Cversion,..) (Same exact version).
  3. Apply a conversion step: ‘csolution convert -s .csolution.yml)

Observed results:

  • Am observing the expected behavior (the GPDSC components definitions with their associated tags (RTE_components.h, Pre_Include_Global, Pre_Include_local) are winning (overriding) the PDSC component definitions only if they have the version greater or equal to the PDSC declared one.
    NB: tests where not applied on files coming from the GPDSC

  • Based on the applied tests, csolution seems to take into consideration the GPDSC file for component override actions.

  • The major challenge that remains and that we are discovering with trials and errors on the GPDSC is that missing attributes in the GPDSC (like the generators tag) or a malformed GPDSC template could cause the tools to crash.

  • Maybe we should enhance the GPDSC Spec by providing a small guideline to describe the list of (PDSC elements) required to exist in the GPDSC template and what is optional:

this would confirm our assumption that every PDSC element described in the spec (https://open-cmsis-pack.github.io/[Open-CMSIS-Pack](https://open-cmsis-pack.github.io/Open-CMSIS-Pack-Spec/main/html/index.html)-Spec/main/html/index.html) is supported via the GPDSC.

@jkrech jkrech added documentation Improvements or additions to documentation and removed bug Something isn't working labels Jun 13, 2024
@jkrech
Copy link
Member

jkrech commented Jun 18, 2024

Summary:

  • csolution crashes on malformed gpdsc files / no built-in schema check
  • update specification listing mandatory elements and attributes in GPDSC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation question Further information is requested
Projects
None yet
Development

No branches or pull requests

3 participants