[RFC] Node Classifications

[lasley]

Something I'm not finding is the concept of a differentiation between specs for the uses in balancing nodes. This is obviously looking forward pretty far, but never hurts to have stuff simmering in your head - gives more time for an inspiring idea or something.

Consider that the typical ideal server specifications would typically be different between the following instance types:

• Database - High RAM, Mid CPU, High Speed & High Capacity Storage
• Application - Mid RAM, High CPU, Low Speed & Low Capacity Storage
• File Store - Mid RAM, Low CPU, Mid Speed & High Capacity Storage

What we need is (list WIP):

• Tags on nodes
• Nodes with tags will only receive Applications of that tag type

[YannickB]

Yes I agree, this will also become quickly a requirement for correctly configuring Docker Swarm.

I think what we need is to manage the label in Docker Swarm https://docs.docker.com/engine/swarm/manage-nodes/, I already thought about it but didn't made any implementation already.

[lasley]

Ah yes I didn't even think about swarm, but this will definitely play into that.

So really this does seem as simple as adding a tag at the Node level. And probably more tags for the services too.

Another concept that we're going to need is a Datacenter. I plan on having nodes in multiple datacenters, and we would obviously not want the Database in East US with the Application in China.

I don't know much about Swarm yet, still on my to-do list in terms of just the bit of skimming I've done. I assume it also has some concept of a datacenter/location?

[YannickB]

Well IMO this shall be managed by the labels (database/USA, compute/USA, database/China, compute/China etc...) :), so we can keep it flexible and avoid complex implementation in Clouder

[lasley]

Hmmm but if we only use tags for the locales, we lose the concept of where the datacenter actually is. This means that we will not be able to add features like GeoIP on our DNS to route to the correct services for where the user is.

I'm obviously thinking way far ahead here, but as far as I know this is not something that any other PaaS provider offers so it would be an excellent competitive advantage I think.

[YannickB]

Hum you're completely right, GeoIP on DNS would be a nice and not too difficult addition. That said it can be simply manage just by adding localisation in node object I think.

We'll see when we'll need this kind of features :)

[pasgou]

Why not considere using the work of tiers developper who have already given solutions, like Rancher.

Clouder is amazing in making images, and Rancher is amazing in docker-compose deployment, with labels on nodes and classifie them for anything you want (you use labels for services in docker-compose files).

With the Rancher API, OVH API, Gandi.net API and Clouder API, we could have an amazing automatic platform to developp and deploy apps in seconds : a app's model for a kind of client deploy in a new container on a new server (if needed) with it's DN created on the fly (if needed), connected to a great database server (the same for hundreds of nodes with replication with the help of some good dockers projects I know and backups with your great work).

Or I don't understand the open source and shared software principles?

[YannickB]

Hello @pasgou,

The main difficulties here are in the design, and how we can integrate with existing software/API.

The spirit of Clouder is, it's the frontend which shall be able to deploy the whole infrastructure from scratch and then administer/billing from its interface. We try to reuse as much as possible, but sometimes it's just not possible or it will complicate the design too much for this incubation step.

That said :

Clouder have the concept of Runner. By default, and to avoid complicating the onboarding, Clouder directly connect to node and deploy services in Swarm mode.
But you can also create a runner module for Kubernetes/Openshift/Rancher/whatever other docker orchestrator, and override the deployment function. You already have some work for Kubernetes and Openshift, work in progress though.

Regarding OVH API/Gandi.net API, and I guess Amazon API/Google Engine API/etc... Clouder is now compatible with libcloud http://libcloud.readthedocs.io/en/latest/, which means Clouder can already deploy nodes in the cloud, and then deploy container with the specified runner. Thanks to libcloud, it can also deploy domain name with external services.

All this work was done two month ago, it's not used in production yet but it's done.

[pasgou]

Yes I know you make a great work and my objective isn't to depreciate it. I want to understand how it's working and sometimes give some advices or questions to brainstorm. Here is the question about make and remake things that others people do each one in his tower everywhere on Earth. That said, I understand that it's not productive to try to be compatible with all others solutions now that you have so progressed. FYI, I'm not a tech nor a dev and probably some complicate concepts escape me. For example what is the difference you make between Clouder that you call a runner and for example Rancher that you call an orchestrator? If the doc were up to date, and the code more documented, perhaps some of my questions weren't asked 😁😁😁 Cordialement Pascal GOUHIER COPAGEX Société d'expertise comptable Membre de la Communauté des Experts en Gestion 8 Boulevard Albert Einstein Parc de la Rivière 44300 NANTES T : 02 40 58 56 49 F : 02 40 95 11 81 http://www.copagex.fr 2017-01-03 10:34 GMT+01:00 Yannick Buron <[email protected]>:

…

Hello @pasgou <https://github.com/pasgou>, The main difficulties here are in the design, and how we can integrate with existing software/API. The spirit of Clouder is, it's the frontend which shall be able to deploy the whole infrastructure from scratch and then administer/billing from its interface. We try to reuse as much as possible, but sometimes it's just not possible or it will complicate the design too much for this incubation step. That said : Clouder have the concept of Runner. By default, and to avoid complicating the onboarding, Clouder directly connect to node and deploy services in Swarm mode. But you can also create a runner module for Kubernetes/Openshift/Rancher/whatever other docker orchestrator, and override the deployment function. You already have some work for Kubernetes and Openshift, work in progress though. Regarding OVH API/Gandi.net API, and I guess Amazon API/Google Engine API/etc... Clouder is now compatible with libcloud http://libcloud.readthedocs.io/en/latest/, which means Clouder can already deploy nodes in the cloud, and then deploy container with the specified runner. Thanks to libcloud, it can also deploy domain name with external services. All this work was done two month ago, it's not used in production yet but it's done. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/clouder-community/clouder/issues/185#issuecomment-270076942>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AIue8WjPIgvez8w7JXgD0xP-Cd1e_ke7ks5rOhYagaJpZM4LNao6> .

[YannickB]

No worries, I was just trying to answer your concern and how we allowed to integrate other components :)

In Clouder runner and orchestrator are the same things. This is external deployer which can be used to deploy containers.

And yeah if only documentation was up to date... We're doing our best ! :)