Merge pull request #18 from Fraunhofer-IMS/release_2_2_0

Release 2.2.0-beta

CITATION.cff

@@ -8,6 +8,35 @@ authors:
   country: DE
   website: https://ims.fraunhofer.de
 title: "AIfES for Arduino"
-version: 2.0.0
-date-released: 2021-07-05
-url: https://github.com/Fraunhofer-IMS/AIfES_for_Arduino
+version: 2.2.0
+date-released: 2024-02-20
+url: https://github.com/Fraunhofer-IMS/AIfES_for_Arduino
+preferred-citation:
+  type: article
+  authors:
+  - family-names: Wulfert
+    given-names: Lars
+  - family-names: Kühnel
+    given-names: Johannes
+  - family-names: Krupp
+    given-names: Lukas
+  - family-names: Viga
+    given-names: Justus
+  - family-names: Wiede
+    given-names: Christian
+  - family-names: Gembaczka
+    given-names: Pierre
+  - family-names: Grabmaier
+    given-names: Anton
+  doi: 10.1109/tpami.2024.3355495
+  identifiers:
+    - type: doi
+      value: 10.1109/tpami.2024.3355495
+    - type: url
+      value: http://dx.doi.org/10.1109/TPAMI.2024.3355495
+    - type: other
+      value: urn:issn:0162-8828
+  title: 'AIfES: A Next-Generation Edge AI Framework'
+  url: http://dx.doi.org/10.1109/TPAMI.2024.3355495
+  journal: IEEE Transactions on Pattern Analysis and Machine Intelligence
+  year: 2024

README.md

@@ -3,11 +3,11 @@
 # AIfES for Arduino®
 
 AIfES (Artificial Intelligence for Embedded Systems) is a platform-independent and standalone AI software framework optimized for embedded systems.
-The Feedforward Neural Networks (FNN) implemented in AIfES can be freely parameterized, trained, modified or reloaded at runtime.
+The Feedforward Neural Networks (FNN) and Convolutional Neural Networks (CNN) implemented in AIfES can be freely parameterized, trained, modified or reloaded at runtime.
 In this version, it is optimized for the Arduino IDE and compatible to almost any Arduino board.
 AIfES is developed in the C programming language and uses only standard libraries based on the GNU Compiler Collection (GCC).
 AIfES thus runs on almost any hardware from 8-bit microcontrollers over Raspberry PI to smartphones or PCs.
-Not only inference of FNN is possible, but also training directly in the device. Furthermore, compatibility to other AI software frameworks such as Keras or TensorFlow is also given.
+Not only inference of FNN and CNN is possible, but also training directly in the device. Furthermore, compatibility to other AI software frameworks such as Keras or TensorFlow is also given.
 
 AIfES was developed by the Fraunhofer Institute for Microelectronic Circuits and Systems IMS. Additional information can be found at <www.aifes.ai>
 
@@ -61,7 +61,14 @@ This standard can speed up inference and training for large FNNs.
 Please read the README.md in https://github.com/Fraunhofer-IMS/AIfES_for_Arduino/tree/main/src/CMSIS for more information on how to add the CMSIS library.
 
 ### Python
-To help you with the quantization of a neural network, we provide some Python tools. You can install them via pip with:
+
+To get you startet we developed the [AIfES-Converter](https://github.com/Fraunhofer-IMS/AIfES-Converter). It can export your FNN from Python and create the suitable AIfES code. 
+Furthermore, it helps you with the quantization of your neural network. 
+You can install it via pip with:
+
+```pip install AIfES-Converter```
+
+The quantization can also be done manually by using the provided Python tools in this repository. You can install them via pip with:
 
 ```pip install https://github.com/Fraunhofer-IMS/AIfES_for_Arduino/raw/main/etc/python/aifes_tools.zip```
 
@@ -101,6 +108,11 @@ The number of neurons and the number of different layers can be adapted individu
 | ELU | ailayer_elu_f32_default() | ailayer_elu_q31_default() | ailayer_elu_q7_default()<br>ailayer_elu_q7_avr_pgm() |
 | Tanh | ailayer_tanh_f32_default() | ailayer_tanh_q31_default() | ailayer_tanh_q7_default()<br>ailayer_tanh_q7_avr_pgm() |
 | Softsign | ailayer_softsign_f32_default() | ailayer_softsign_q31_default()| ailayer_softsign_q7_default()<br>ailayer_softsign_q7_avr_pgm() |
+| Conv2D | ailayer_conv2d_f32_default() |  |  |
+| Batch Normalization | ailayer_batch_norm_f32_default() |  |  |
+| MaxPool2D | ailayer_maxpool2d_f32_default() |  |  |
+| Reshape | ailayer_reshape_f32_default() |  |  |
+| Flatten | ailayer_flatten_f32_default() |  |  |
 
 **Training layer**
 
@@ -115,6 +127,11 @@ The number of neurons and the number of different layers can be adapted individu
 | ELU | ailayer_elu_f32_default() | ailayer_elu_q31_default()|         |
 | Tanh | ailayer_tanh_f32_default() | ailayer_tanh_q31_default() |         |
 | Softsign | ailayer_softsign_f32_default() | ailayer_softsign_q31_default() |         |
+| Conv2D | ailayer_conv2d_f32_default() |  |  |
+| Batch Normalization | ailayer_batch_norm_f32_default() |  |  |
+| MaxPool2D | ailayer_maxpool2d_f32_default() |  |  |
+| Reshape | ailayer_reshape_f32_default() |  |  |
+| Flatten | ailayer_flatten_f32_default() |  |  |
 
 **Loss:**
 
@@ -138,8 +155,30 @@ Alternatively, the manual download is also possible:
 Download the AIfES repository as a ZIP archive and follow these instructions:
 <https://www.arduino.cc/en/guide/libraries>
 
-## Roadmap
-The AIfES team at Fraunhofer IMS is constantly working on new features and network types. The next feature you can look forward to is:
+## Citation
+If you use this software in your work please cite it.
+
+For your scientific work you can cite the following paper:
+
+Plain Text:
+``` 
+L. Wulfert et al., "AIfES: A Next-Generation Edge AI Framework," in IEEE Transactions on Pattern Analysis and Machine Intelligence, doi: 10.1109/TPAMI.2024.3355495.
+keywords: {Training;Data models;Artificial intelligence;Support vector machines;Hardware acceleration;Libraries;Performance evaluation;Machine Learning Framework;Edge AI Framework;On-Device Training;Embedded Systems;Resource-Constrained Devices;TinyML}
+```
+
+BibTex:
+```
+@ARTICLE{10403985,
+  author={Wulfert, Lars and Kühnel, Johannes and Krupp, Lukas and Viga, Justus and Wiede, Christian and Gembaczka, Pierre and Grabmaier, Anton},
+  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence}, 
+  title={AIfES: A Next-Generation Edge AI Framework}, 
+  year={2024},
+  volume={},
+  number={},
+  pages={1-16},
+  keywords={Training;Data models;Artificial intelligence;Support vector machines;Hardware acceleration;Libraries;Performance evaluation;Machine Learning Framework;Edge AI Framework;On-Device Training;Embedded Systems;Resource-Constrained Devices;TinyML},
+  doi={10.1109/TPAMI.2024.3355495}}
+```
 
-### Convolutional Neural Networks (ConvNet) 
-Soon, classical ConvNets can be executed and trained in AIfES.
+## Roadmap
+The AIfES team at Fraunhofer IMS is constantly working on new features and network types.

docs/_tutorial_inference_f32.html

@@ -223,7 +223,7 @@ <h2><a class="anchor" id="autotoc_md10"></a>
 <div class="ttc" id="astructailayer__input_html"><div class="ttname"><a href="structailayer__input.html">ailayer_input</a></div><div class="ttdoc">General Input layer  structure.</div><div class="ttdef"><b>Definition:</b> ailayer_input.h:39</div></div>
 <div class="ttc" id="astructailayer__leaky__relu__f32_html"><div class="ttname"><a href="structailayer__leaky__relu__f32.html">ailayer_leaky_relu_f32</a></div><div class="ttdoc">Data-type specific Leaky ReLU layer struct for F32 .</div><div class="ttdef"><b>Definition:</b> ailayer_leaky_relu_default.h:51</div></div>
 <div class="ttc" id="astructailayer__sigmoid_html"><div class="ttname"><a href="structailayer__sigmoid.html">ailayer_sigmoid</a></div><div class="ttdoc">General Sigmoid layer  struct.</div><div class="ttdef"><b>Definition:</b> ailayer_sigmoid.h:47</div></div>
-<div class="ttc" id="astructaimodel_html"><div class="ttname"><a href="structaimodel.html">aimodel</a></div><div class="ttdoc">AIfES artificial neural network model.</div><div class="ttdef"><b>Definition:</b> aifes_core.h:178</div></div>
+<div class="ttc" id="astructaimodel_html"><div class="ttname"><a href="structaimodel.html">aimodel</a></div><div class="ttdoc">AIfES artificial neural network model.</div><div class="ttdef"><b>Definition:</b> aifes_core.h:181</div></div>
 </div><!-- fragment --><p>We use the initializer macros with the "_M" at the end, because we need to set our parameters (like the weights) to the layers.</p>
 <h2><a class="anchor" id="autotoc_md11"></a>
 Connection and initialization of the layers</h2>
@@ -244,9 +244,9 @@ <h2><a class="anchor" id="autotoc_md11"></a>
 <div class="ttc" id="aailayer__input__default_8h_html_af7bb369ff05cf9cca9fda5765a951c79"><div class="ttname"><a href="ailayer__input__default_8h.html#af7bb369ff05cf9cca9fda5765a951c79">ailayer_input_f32_default</a></div><div class="ttdeci">ailayer_t * ailayer_input_f32_default(ailayer_input_f32_t *layer)</div><div class="ttdoc">Initializes and connect an Input layer  with the F32  default implementation.</div></div>
 <div class="ttc" id="aailayer__leaky__relu__default_8h_html_aecb15e9008d56b41a8370d9f38cde60c"><div class="ttname"><a href="ailayer__leaky__relu__default_8h.html#aecb15e9008d56b41a8370d9f38cde60c">ailayer_leaky_relu_f32_default</a></div><div class="ttdeci">ailayer_t * ailayer_leaky_relu_f32_default(ailayer_leaky_relu_f32_t *layer, ailayer_t *input_layer)</div><div class="ttdoc">Initializes and connect a Leaky ReLU layer  with the F32  default implementation.</div></div>
 <div class="ttc" id="aailayer__sigmoid__default_8h_html_a1ec45b121e81b85e2109f0072d46f602"><div class="ttname"><a href="ailayer__sigmoid__default_8h.html#a1ec45b121e81b85e2109f0072d46f602">ailayer_sigmoid_f32_default</a></div><div class="ttdeci">ailayer_t * ailayer_sigmoid_f32_default(ailayer_sigmoid_f32_t *layer, ailayer_t *input_layer)</div><div class="ttdoc">Initializes and connect a Sigmoid layer  with the F32  default implementation.</div></div>
-<div class="ttc" id="astructailayer_html"><div class="ttname"><a href="structailayer.html">ailayer</a></div><div class="ttdoc">AIfES layer interface.</div><div class="ttdef"><b>Definition:</b> aifes_core.h:249</div></div>
-<div class="ttc" id="astructaimodel_html_a708a94e69112ad215b2b52da2238a711"><div class="ttname"><a href="structaimodel.html#a708a94e69112ad215b2b52da2238a711">aimodel::input_layer</a></div><div class="ttdeci">ailayer_t * input_layer</div><div class="ttdoc">Input layer of the model that gets the input data.</div><div class="ttdef"><b>Definition:</b> aifes_core.h:179</div></div>
-<div class="ttc" id="astructaimodel_html_a7c7ad89e7d15631b3f5893b8f19030ef"><div class="ttname"><a href="structaimodel.html#a7c7ad89e7d15631b3f5893b8f19030ef">aimodel::output_layer</a></div><div class="ttdeci">ailayer_t * output_layer</div><div class="ttdoc">Output layer of the model.</div><div class="ttdef"><b>Definition:</b> aifes_core.h:180</div></div>
+<div class="ttc" id="astructailayer_html"><div class="ttname"><a href="structailayer.html">ailayer</a></div><div class="ttdoc">AIfES layer interface.</div><div class="ttdef"><b>Definition:</b> aifes_core.h:252</div></div>
+<div class="ttc" id="astructaimodel_html_a708a94e69112ad215b2b52da2238a711"><div class="ttname"><a href="structaimodel.html#a708a94e69112ad215b2b52da2238a711">aimodel::input_layer</a></div><div class="ttdeci">ailayer_t * input_layer</div><div class="ttdoc">Input layer of the model that gets the input data.</div><div class="ttdef"><b>Definition:</b> aifes_core.h:182</div></div>
+<div class="ttc" id="astructaimodel_html_a7c7ad89e7d15631b3f5893b8f19030ef"><div class="ttname"><a href="structaimodel.html#a7c7ad89e7d15631b3f5893b8f19030ef">aimodel::output_layer</a></div><div class="ttdeci">ailayer_t * output_layer</div><div class="ttdoc">Output layer of the model.</div><div class="ttdef"><b>Definition:</b> aifes_core.h:183</div></div>
 </div><!-- fragment --><h2><a class="anchor" id="autotoc_md12"></a>
 Print the layer structure to the console</h2>
 <p>To see the structure of your created model, you can print a model summary to the console </p><div class="fragment"><div class="line">aiprintf(<span class="stringliteral">&quot;\n-------------- Model structure ---------------\n&quot;</span>);</div>
@@ -280,14 +280,14 @@ <h2><a class="anchor" id="autotoc_md14"></a>
 <div class="line"><span class="keywordtype">float</span> in_data[1*3] = {1.0f, 0.0f, 0.0f};</div>
 <div class="line"><a class="code" href="structaitensor.html">aitensor_t</a> in = AITENSOR_2D_F32(in_shape, in_data);</div>
 <div class="ttc" id="astructaitensor_html"><div class="ttname"><a href="structaitensor.html">aitensor</a></div><div class="ttdoc">A tensor in AIfES.</div><div class="ttdef"><b>Definition:</b> aifes_math.h:89</div></div>
-</div><!-- fragment --><p>Now everything is ready to perform the actual inference. For this you can use the function <a class="el" href="aialgo__sequential__inference_8h.html#aafaea85e93de468925ada90c77494eec" title="Perform an inference on the model / Run the model.">aialgo_inference_model()</a>.</p>
+</div><!-- fragment --><p>Now everything is ready to perform the actual inference. For this you can use the function <a class="el" href="aialgo__sequential__inference_8h.html#a295cb2bb6c3fefc478042f66e067e090" title="Perform an inference on the model / Run the model.">aialgo_inference_model()</a>.</p>
 <div class="fragment"><div class="line"><span class="comment">// Create an empty tensor for the inference results</span></div>
 <div class="line">uint16_t out_shape[2] = {1, 2};</div>
 <div class="line"><span class="keywordtype">float</span> out_data[1*2];</div>
 <div class="line"><a class="code" href="structaitensor.html">aitensor_t</a> out = AITENSOR_2D_F32(out_shape, out_data);</div>
 <div class="line"> </div>
-<div class="line"><a class="code" href="aialgo__sequential__inference_8h.html#aafaea85e93de468925ada90c77494eec">aialgo_inference_model</a>(&amp;model, &amp;in, &amp;out);</div>
-<div class="ttc" id="aaialgo__sequential__inference_8h_html_aafaea85e93de468925ada90c77494eec"><div class="ttname"><a href="aialgo__sequential__inference_8h.html#aafaea85e93de468925ada90c77494eec">aialgo_inference_model</a></div><div class="ttdeci">aitensor_t * aialgo_inference_model(aimodel_t *model, aitensor_t *input_data, aitensor_t *output_data)</div><div class="ttdoc">Perform an inference on the model / Run the model.</div></div>
+<div class="line"><a class="code" href="aialgo__sequential__inference_8h.html#a295cb2bb6c3fefc478042f66e067e090">aialgo_inference_model</a>(&amp;model, &amp;in, &amp;out);</div>
+<div class="ttc" id="aaialgo__sequential__inference_8h_html_a295cb2bb6c3fefc478042f66e067e090"><div class="ttname"><a href="aialgo__sequential__inference_8h.html#a295cb2bb6c3fefc478042f66e067e090">aialgo_inference_model</a></div><div class="ttdeci">uint8_t aialgo_inference_model(aimodel_t *model, aitensor_t *input_data, aitensor_t *output_data)</div><div class="ttdoc">Perform an inference on the model / Run the model.</div></div>
 </div><!-- fragment --><p>Alternative you can also do the inference without creating an empty tensor for the result with the function <a class="el" href="aialgo__sequential__inference_8h.html#a4655caab3051cc837312c286fbe4789a" title="Perform a forward pass on the model.">aialgo_forward_model()</a>. The results of this function are stored in the inference memory. If you want to perform another inference or delete the inference memory, you have to save the results first to another tensor / array. Otherwise you will loose the data. </p><div class="fragment"><div class="line"><a class="code" href="structaitensor.html">aitensor_t</a> *y = <a class="code" href="aialgo__sequential__inference_8h.html#a4655caab3051cc837312c286fbe4789a">aialgo_forward_model</a>(&amp;model, &amp;in);</div>
 <div class="ttc" id="aaialgo__sequential__inference_8h_html_a4655caab3051cc837312c286fbe4789a"><div class="ttname"><a href="aialgo__sequential__inference_8h.html#a4655caab3051cc837312c286fbe4789a">aialgo_forward_model</a></div><div class="ttdeci">aitensor_t * aialgo_forward_model(aimodel_t *model, aitensor_t *input_data)</div><div class="ttdoc">Perform a forward pass on the model.</div></div>
 </div><!-- fragment --><p>Afterwards you can print the results to the console for debugging purposes: </p><div class="fragment"><div class="line">aiprintf(<span class="stringliteral">&quot;input:\n&quot;</span>);</div>