Skip to content

Commit

Permalink
api: support roles.httpd integration
Browse files Browse the repository at this point in the history 
Since the `httpd` role was released this role has been needed support
for this feature.

After the patch metrics-export-role supports an `httpd` role. It is
possible to determine a list of servers that role will reuse in the
following configuration.

Closes #15
  • Loading branch information
@themilchenko
themilchenko committed Oct 2, 2024
1 parent 4c7758c commit d439bcf
Show file tree
Hide file tree
Showing 8 changed files with 1,345 additions and 204 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added
- Introduce latency observation for http endpoint (#17).
- Support `roles.httpd` integration (#15).

### Fixed

Expand Down
72 changes: 67 additions & 5 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,8 @@ tt build
Be careful, it is better to use a latest release version.

2. Enable and [configure](https://www.tarantool.io/en/doc/latest/concepts/configuration/)
the `roles.metrics-export` role for a Tarantool 3 instance.
the `roles.metrics-export` role for a Tarantool 3 instance. Use [httpd role](https://github.com/tarantool/http?tab=readme-ov-file#roles)
or `listen` field in to configure server instances. See below to get more detailed information about it.

```yaml
groups:
Expand All @@ -37,10 +38,22 @@ groups:
replicaset-001:
instances:
master:
roles: [roles.metrics-export]
roles: [roles.httpd, roles.metrics-export]
roles_cfg:
roles.httpd:
default:
- listen: '127.0.0.1:8083'
additional:
- listen: 8084
roles.metrics-export:
http:
- endpoints:
- path: /metrics/json
format: json
- server: 'additional'
endpoints:
- path: /metrics/prometheus
format: prometheus
- listen: 8081
endpoints:
- path: /metrics/json
Expand All @@ -55,9 +68,13 @@ groups:
format: json
```
In the example above, we configure two HTTP servers on `0.0.0.0:8081` and
`my_host:8082`. The servers will be running on the `master` Tarantool
instance.
In the example above, we configure four HTTP servers. There are serveral server fields:
* first with `server` field which refers to an `additional` server in the `httpd` role;
* the next one, with no info about server, is configured with `default` name in `httpd` config;
* and the last two `listen` fields (`0.0.0.0:8081` and `my_host:8082`) that are listed directly.

The servers will be running on the `master` Tarantool instance.

Each server has two endpoints:

Expand Down Expand Up @@ -103,10 +120,46 @@ set `metrics.enabled` to `true`:

For now only `json` and `prometheus` formats are supported.

### Integration with httpd role

Use [httpd role](https://github.com/tarantool/http?tab=readme-ov-file#roles) as well.
To enable it, you need to fill `server` field with name that was configured in `roles.httpd` block
instead of `listen` like it was earlier. To configure `httpd` role you need to write block in roles_cfg
section:

```yaml
roles_cfg:
roles.httpd:
default:
- listen: 8081
additional:
- listen: '127.0.0.1:8082'
```

After it you can use `server` name in `roles.metrics-export` block. If `server` and `listen` names
wasn't provided, the `default` server from `httpd` role configuration will be used:

```yaml
roles.metrics-export:
http:
- server: 'additional'
endpoints:
...
- endpoints:
...
```

So now it is possible to mix `server` and `listen` parameteres.

Let's put it all together now:

```yaml
roles_cfg:
roles.httpd:
default:
- listen: 8081
additional:
- listen: '127.0.0.1:8082'
roles.metrics-export:
http:
- listen: 8081
Expand All @@ -121,6 +174,15 @@ roles_cfg:
format: json
metrics:
enabled: true
- server: 'additional'
endpoints:
- path: /metrics
format: json
- endpoints:
- path: /metrics
format: prometheus
metrics:
enabled: true
```

With this configuration, metrics can be obtained on this machine with the
Expand Down
141 changes: 103 additions & 38 deletions roles/metrics-export.lua
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
local urilib = require("uri")
local http_server = require('http.server')
local httpd_role = require('roles.httpd')

local M = {}

Expand All @@ -23,19 +24,6 @@ local function is_array(tbl)
return true
end

local function delete_route(httpd, name)
local route = assert(httpd.iroutes[name])
httpd.iroutes[name] = nil
table.remove(httpd.routes, route)

-- Update httpd.iroutes numeration.
for n, r in ipairs(httpd.routes) do
if r.name then
httpd.iroutes[r.name] = n
end
end
end

-- Removes extra '/' from start and end of the path to avoid paths duplication.
local function remove_side_slashes(path)
if path:startswith('/') then
Expand All @@ -48,9 +36,6 @@ local function remove_side_slashes(path)
end

local function parse_listen(listen)
if listen == nil then
return nil, nil, "must exist"
end
if type(listen) ~= "string" and type(listen) ~= "number" then
return nil, nil, "must be a string or a number, got " .. type(listen)
end
Expand Down Expand Up @@ -170,14 +155,39 @@ local function validate_http_endpoint(endpoint)
end
end

-- check_server_httpd_role validates that httpd configuration and provided name exists.
local function check_server_httpd_role(server)
local httpd_roles_cfg = (require("config"):get("roles_cfg") or {})['roles.httpd']
if httpd_roles_cfg == nil then
error("there is no configuration for httpd role", 4)
end
if httpd_roles_cfg[server] == nil then
error(("server with name %s not found in httpd role config"):format(server), 4)
end
end

local function validate_http_node(node)
if type(node) ~= "table" then
error("http configuration node must be a table, got " .. type(node), 3)
end

local _, _, err = parse_listen(node.listen)
if err ~= nil then
error("failed to parse http 'listen' param: " .. err, 3)
if node.server ~= nil then
if type(node.server) ~= 'string' then
error("server configuration sould be a string, got " .. type(node.server), 3)
end

if node.listen ~= nil then
error("it is not possible to provide 'server' and 'listen' blocks simultaneously", 3)
end

check_server_httpd_role(node.server)
elseif node.listen ~= nil then
local _, _, err = parse_listen(node.listen)
if err ~= nil then
error("failed to parse http 'listen' param: " .. err, 3)
end
else
check_server_httpd_role(httpd_role.DEFAULT_SERVER_NAME)
end

node.endpoints = node.endpoints or {}
Expand Down Expand Up @@ -221,16 +231,31 @@ local function validate_http(conf)
end

for i, nodei in ipairs(conf) do
local hosti, porti, erri = parse_listen(nodei.listen)
assert(erri == nil) -- We should already successfully parse the URI.
local listen_address, server_name = nodei.listen, nodei.server
local hosti, porti = nil, nil
if listen_address ~= nil then
local erri
hosti, porti, erri = parse_listen(listen_address)
assert(erri == nil) -- We should already successfully parse the URI.
end
for j, nodej in ipairs(conf) do
if i ~= j then
local hostj, portj, errj = parse_listen(nodej.listen)
listen_address = nodej.listen
if listen_address == nil then
if server_name ~= nil and server_name == nodej.server or
server_name == httpd_role.DEFAULT_SERVER_NAME and nodej.server == nil or
server_name == nil and nodej.server == httpd_role.DEFAULT_SERVER_NAME then
error("server names must have different targets in httpd", 2)
end
goto continue
end
local hostj, portj, errj = parse_listen(listen_address)
assert(errj == nil) -- The same.
if hosti == hostj and porti == portj then
error("http configuration nodes must have different listen targets", 2)
end
end
::continue::
end
end
end
Expand Down Expand Up @@ -258,24 +283,52 @@ local function apply_http(conf)
local enabled = {}
for _, node in ipairs(conf) do
if #(node.endpoints or {}) > 0 then
local host, port, err = parse_listen(node.listen)
if err ~= nil then
error("failed to parse URI: " .. err, 2)
local host, port, target
if node.server ~= nil then
target = {
value = node.server,
is_httpd_role = true,
}
elseif node.listen ~= nil then
local err
host, port, err = parse_listen(node.listen)
if err ~= nil then
error("failed to parse URI: " .. err, 2)
end
target = {
value = node.listen,
is_httpd_role = false,
}
else
target = {
value = httpd_role.DEFAULT_SERVER_NAME,
is_httpd_role = true,
}
end
local listen = node.listen

http_servers = http_servers or {}
enabled[listen] = true
-- Since the 'listen' and 'server' names of other servers in the config may be
-- the same, we create a unique string concatenating the key name and information
-- about whether it is an httpd key or not.
enabled[tostring(target.value) .. tostring(target.is_httpd_role)] = true

if http_servers[tostring(target.value) .. tostring(target.is_httpd_role)] == nil then
local httpd
if node.listen ~= nil then
httpd = http_server.new(host, port)
httpd:start()
else
httpd = httpd_role.get_server(target.value)
end

if http_servers[listen] == nil then
local httpd = http_server.new(host, port)
httpd:start()
http_servers[listen] = {
http_servers[tostring(target.value) .. tostring(target.is_httpd_role)] = {
httpd = httpd,
routes = {},
is_httpd_role = target.is_httpd_role,
}
end
local server = http_servers[listen]

local server = http_servers[tostring(target.value) .. tostring(target.is_httpd_role)]
local httpd = server.httpd
local old_routes = server.routes

Expand All @@ -291,7 +344,7 @@ local function apply_http(conf)
-- Remove old routes.
for path, e in pairs(old_routes) do
if new_routes[path] == nil or not routes_equal(e, new_routes[path]) then
delete_route(httpd, path)
httpd:delete(path)
old_routes[path] = nil
end
end
Expand All @@ -315,17 +368,29 @@ local function apply_http(conf)
end
end

for listen, server in pairs(http_servers) do
if not enabled[listen] then
server.httpd:stop()
http_servers[listen] = nil
for target, server in pairs(http_servers) do
if not enabled[target] then
if server.is_httpd_role then
for path, _ in pairs(server.routes) do
server.httpd:delete(path)
end
else
server.httpd:stop()
end
http_servers[target] = nil
end
end
end

local function stop_http()
for _, server in pairs(http_servers or {}) do
server.httpd:stop()
if server.is_httpd_role then
for path, _ in pairs(server.routes) do
server.httpd:delete(path)
end
else
server.httpd:stop()
end
end
http_servers = nil
end
Expand Down
22 changes: 21 additions & 1 deletion test/entrypoint/config.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,13 @@ groups:
replicaset-001:
instances:
master:
roles: [roles.metrics-export]
roles: [roles.httpd, roles.metrics-export]
roles_cfg:
roles.httpd:
default:
listen: 8085
additional:
listen: '127.0.0.1:8086'
roles.metrics-export:
http:
- listen: 8081
Expand All @@ -29,6 +34,21 @@ groups:
format: prometheus
metrics:
enabled: true
- endpoints:
- path: /metrics/prometheus
format: prometheus
- path: /metrics/json
format: json
- server: 'additional'
endpoints:
- path: /metrics/prometheus
format: prometheus
- path: /metrics/json
format: json
- path: /metrics/observed/prometheus/1
format: prometheus
metrics:
enabled: true
iproto:
listen:
- uri: '127.0.0.1:3313'
Expand Down
Loading

0 comments on commit d439bcf

Please sign in to comment.