Backend design: whence the APIs?

[ziz]

Currently we are contemplating having

• A Vue frontend
• A Node backend to support the Vue frontend
• A Python backend to communicate with the various controllers

There exist initial API routes in the Node backend (and in design.txt, which we want to go away).

Most (all?) of these seem likely to end up being implemented as either "call the same API in the Python backend" or "call a slightly different API in the Python backend".

Advantage: This lets the Node app stand alone for development and testing.

Disadvantage: We have to duplicate all the interfaces and contracts for APIs we're calling in JS and Python. We can't even keep them consistent (I am not free from sin) between design.txt and api.js; Ada Lovelace help us if we have actual implementations to keep consistent, especially if when we need to debug on-playa.

Assuming I'm not missing something there, I propose cutting out the middleman; define APIs in the Python backend and call to them directly where possible.

Thoughts/concerns/alternate proposals?

[cstigler]

@ziz I think my understanding of the current plan was different, so thank you for raising this!

My thought had been that there would be one server that takes requests from the Vue frontend and then communicates out to the various controllers. I don't see the point in separating that into two servers (sounds like you don't either).

My thought had been that the Node server would be that (the single backend that supports the Vue frontend and then communicates out).

If you'd like to get rid of Node and make it a Python server instead, that's fine with me as long as you're writing all of the backend code! I am much more competent with Node than Python so would prefer it if I'm expecting to write much there.

[ziz]

Hurrah for finding communication fails early!

If I'm understanding correctly, you (@cstigler) were envisioning something like (rendered):

  digraph G {
    human -> vue [weight=8,label="Click something"];

    vue -> nodejs [label="HTTP (REST)"];
    nodejs -> firepy [label="sockets/http?"];
    nodejs -> ledpy [label="sockets/http?"];
    nodejs -> soundpy [label="sockets/http?"];

    firepy -> rpis [label="multicast - bang?"];
    ledpy -> rpis [label="multicast - NOT bang?"];
    soundpy -> rpis [label="multicast - NOT bang?"];


    human [shape=box,label="Human"];
    subgraph cluster_admin_rpi {
        label="Admin RPi";
        color=blue;
        vue [label="Vue page"];
        nodejs [label="Node JS app"];
        firepy [label="Fire controller py app"];
        ledpy [label="LED controller py app"];
        soundpy [label="Sound controller py app"];
    }
    rpis [label="Arduinos / poofer boards / sound controllers / etc in sculpture"];
  }

Is that reasonably accurate?

I am roughly proposing (rendered):

  digraph G {
    human -> vue [weight=8,label="Click something"];

    vue -> firepy [label="HTTP (REST)"];
    vue -> ledpy [label="HTTP (REST)"];
    vue -> soundpy [label="HTTP (REST)"];

    vue -> nodejs [label="HTTP (REST)",weight=2];
    
    firepy -> rpis [label="multicast - bang?"];
    ledpy -> rpis [label="multicast - NOT bang?"];
    soundpy -> rpis [label="multicast - NOT bang?"];

    human [shape=box,label="Human"];
    subgraph cluster_admin_rpi {
        label="Admin RPi";
        subgraph javascript {
            rank=same;
            nodejs [label="Node JS app PROBABLY UNNEEDED",group="app"];
            vue [label="Vue page",group="app"];
        }
        firepy [label="Fire controller py app"];
        ledpy [label="LED controller py app"];
        soundpy [label="Sound controller py app"];
    }
    rpis [label="Arduinos / poofer boards / sound controllers / etc in sculpture"];

  }

(Where I have said the node JS app is "probably unneeded", possibly what I mean is "I don't know if it is more pragmatic for it to do aggregation of calls to the py controllers or for that to happen on the front end, but it shouldn't just be passing an essentially unmodified request from the front end to the py controllers)

[cstigler]

Ah, I see. Yep, your top diagram captures what I was envisioning pretty accurately. It looks like the main change in your proposal is that we entirely eliminate a single centralized controller, and instead talk directly to each individual controller.

I'm mostly fine either way, but want to note a few tradeoffs of eliminating the central controller:

• It means that each controller would have to fulfill the responsibilities of a UI-facing web server: sending well-formatted REST error responses (not just communicating via sockets etc), setting the right headers, etc.
• We wouldn't have an obvious place to serve the small amount of static assets for the web interface
• We couldn't aggregate calls to different controllers together (ex: if the UI wants to "get the current status of everything", that must be done with three calls on the UI vs one call to the Node server that could aggregate)
• If we ever end up with other areas (non-UI) where we want a central controller to coordinate requests between the different controllers (ex: coordinate the lights and fire changing in harmony), there wouldn't be an obvious central place to coordinate those

I suspect @cswales may have some wise thoughts from past experience...

[ziz]

You make good arguments in favor of the top diagram (having an admin server to be at least a thin wrapper), but I'm still concerned about defining essentially identical API interfaces twice.

I suppose it also depends somewhat on what the flame/sound/light controllers look like.

In Noetica, it appears that the flame controller was also the primary static content/UI webserver, which had passthrough routes to the hydraulics controller (which was itself a UI/app-facing web server itself - but without static content, just API routes).

We could possibly make more aggressive use of passthrough routes on the Node app? Say, let everything under /flames/ get handed off to the flame controller (possibly with a little bit of wrapping on the Node side to support the Vue interface), and likewise for /lights/ and /sound/.

[cstigler]

Yep, that sounds like a good plan to me! Still requires that all servers send equivalently-formatted responses, but I imagine that's not too much work.

[cswales]

I'm in favor of having each of the servers send out their own REST responses. Fewer interfaces to define, and fewer layers, as Justin points out. (I'm also smarting from having badly miscalculated on how much time multiple layers of interfaces was going to take me on last year's project, Charlie's Crown. I do not want to spend hours on the playa plumbing the same functionality through three layers of interfaces ever again.)
Remember though, that there are multiple LED controllers and sound controllers, so there's going to have to be considerably more than three calls made to get global status. But - it's a local wired network, so you're pretty much guaranteed of getting a response very quickly. (There's only one flame controller, and there's only one firefly-led controller. Those will both probably live on the same pi as the webserver, but if everyone communicates with REST, who really cares?)