Oooops, starting the server takes very long

[mschnell1]

I moved the EOS directory from the local SSD (on Windows) to my NAS (which provides automatic nightly backups). The NAS is connected by GBIT Ethernet and usually decently fast.
When I now start the server by .venv\\Scripts\\fastapi run --port 8503 src/akkudoktoreos/server/fastapi_server.py, "Searching for package file structure from directories with init.py files" takes several minutes.
Is that normal / desirable ?

[Lasall]

Can you reproduce this problem with a minimal fastapi-Setup? https://fastapi.tiangolo.com/#example

[mschnell1]

OK. Hence something like SOAP.
This sounds like a device<->device communication.
But what I see when connecting to the server is a GUI in the browser. (device<->user communication)
Hence I am confused.

[Lasall]

At the /docs endpoint you find some service definitions, similar to WSDL.

[mschnell1]

No idea at all what "service definitions" are. (OK, I looked up WSDL and found that it is for defining Service definitions which does not help me a bit)

No answer to the question if this is a device<->device communication or something like a GUI. I don't understand the purpose of FastAPI within the project, and the FastAPI server (if that really is this thing) is the only snippet I see up to now.

OK. I give up.

(No answer to #340 about if there might be something I might be able to contribute, so obviously I am at the wrong place here.)

[Lasall]

Sorry to make you feel frustrated. Maybe at a later stage of the project with better documentation you can give it a try again.

In general https://github.com/Akkudoktor-EOS/EOS/issues lists a few issues tagged with "good first issue" that might be a good entrypoint to project development as they address mostly localized problems. Otherwise first documentation of the development branch feature/config-overhaul can be found at https://akkudoktor-eos.readthedocs.io/en/latest/ (including some developer documentation).

A vision and rough roadmap can be found at https://github.com/Akkudoktor-EOS/EOS/wiki/Roadmap-and-Vision

[mschnell1]

I am researching if I might be able to contribute. Later or even right now.
Taking another look ASAP to the links you provided ...

[NicoMittenzwey]

This is typical Python behavior especially if your NAS uses HDDs instead of SSDs. Python relies on a lot of small files. Even if the NAS and the network can provide a decent bandwidth, the latency is the bottleneck. Starting the server for example creates 10000 file metadata operations alone which all result in a single (typically random) read operation on the NAS. On top of that the content of the files have to be read and transferred. So the solution here would be to have your files on your local SSD and sync them to the NAS drive for backup periodically.

[mschnell1]

So the solution here would be to have your files on your local SSD and sync them to the NAS drive for backup periodically.

I suppose once I might create some useful stuff, I can either use the standard EOS GitHub repository or use a .git folder on the NAS for content management.

[mschnell1]

This project relies heavily on existing data analysis / machine learning libraries (the "GByte py and pyc files") for the optimizations and predictions. It's way beyond the scope of this project to port all of those existing libraries to Rust. And I think it was never intended to run the optimizations and predictions on a embedded device as those devices usually lag the needed resources in compute power as well as disk space.

I see. (of course "rewrite in Rust" was a joke,I do know that the advantage of using Python is the early availability of rather mature libraries)
That is why I said (multiple) "products". I do understand that one "product" is the kind of research / global-and-local optimization tool to be run by the user on request.
But I do suppose that some embedded unit - running all day - to control the local infrastructure (collecting data and controlling devices), which would be fed with the research results should / could be another related "product". Same should at least be able to run on a RasPI (some 50€), but even better if only a PICO (some 5€ and much lower power consumption) would be possible. That is why I started this discussion ....

[mschnell1]

And the embedded "product" also should have a web server for monitoring and configuring.
And it should be accessible by the "optimizations and predictions" products via TCP/IP to feed collected data to same and to receive optimized control parameters from same.
Am I right assuming that the plan is something along these lines ?

[b0661]

And the embedded "product" also should have a web server for monitoring and configuring.

There is already one. You find it in fasthtml_server.py and it actually displays the configuration that you see when you access localhost:8503 and your configuration automatically starts the fasthtml_server.

And it should be accessible by the "optimizations and predictions" products via TCP/IP to feed collected data to same and to receive optimized control parameters from same.

This is done by the REST server that is in fastapi_server.py. You access it by the same localhost:8503 but only certain pathes are for REST PUT and POST.

Am I right assuming that the plan is something along these lines ?

Yes. See above.

[mschnell1]

I do not yet understand enough of the project structure, and can't easily get up to speed due to lack of experience with Python to understand all this,

Question: I seem to understand that an appropriate "product" / "distribution" of the software is supposed to run on a RasPi all day, controlling hardware and collecting Data, and temporarily will communicate with a "optimization / prediction Server" (a different "product" / "distribution" , supposedly running on a Desktop or similar), that gets parameters by the user and does complex calculations.

Are there already lists of the hardware that is to be accessed, protocols to be used for that, and interface hardware ("caps" in Raspi speak) to be available ?

When controlling a distributed installation (Akku, Smart Meter, Heat Pump, ...) maybe multiple RasPis (or other "IOT" devices) in a network might be sensible

Thanks again!

[mschnell1]

Oooops .... This seem not desirable at all to me.
I don't know the structure of the "EOS" project and the possible "distributions", yet.
I think, one of the possible "products" should be an embedded software thingy being able to run on (e.g.) a PICO2 all day long and control devices (solar/Akku stuff, heat pump, smart meter, ...). Here we would not have infinite amount of "disk" or in fact no standard file system at all..

[mschnell1]

Half a GByte py and pyc files as "site-packages seem really huge.

[mschnell1]

Sorry. I have been doing embedded software all my live, and hence this is my primary focus.

[b0661]

Think of it as Linux Embedded, which definitely works. Maybe not on a Pico, but on a normal Raspy.

[mschnell1]

Not perfect,but definitively just fine !