-
Notifications
You must be signed in to change notification settings - Fork 5
/
Copy pathTODO.txt
102 lines (96 loc) · 5.94 KB
/
TODO.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
Always:
- keep direct dependencies up to date
- (go list -m -u all)
- go get -u ./... && go mod tidy
- follow new libvirt.org/go/libvirt major releases, see :
- https://libvirt.org/news.html
- https://pkg.go.dev/libvirt.org/go/libvirt
Resource leaks:
- check for remaining map race-conditions
- check that the switch to Context fixes all known resource leaks
- mulchd "outage" of 2020-07-22:
- investigate all the preparePipes.funcX found in the stacktrace (see m2 log)
- check for a possible deadlock / missing timeout in the message hub? (same)
- maybe fixed by the switch to Context of run.go
- still some hanging SSH shown in status (try with a client reboot?)
- scheduleSSHKeepAlives() does its job, the issue's upper in the stack (runChannels never ends?)
- added goroutine pprof profile to help
- seen recently during a "foreach-vm ssh loop" script when mashing CTRL+c!
- also when restarting nosee? (but in this case, "ss" showed real tcp connections)
- mulchd memory usage (10+GB)
- normal for Go 1.12 → 1.15 runtimes, memory is released to the kernel if
it needs it (MADV_FREE), but but RSS does not capture this information
- Go 1.16 reverted to MADV_DONTNEED, so this is not a MADV_FREE false positive
- heap profile show a very small memory consumption, comparatively
- seems libvirt related! (Streams & Volume transferts) - anonymous allocations
Short term:
- SSH proxy: can we write an error to the client on early error? (is it a good idea anyway?)
- see SSHProxy.serveProxy()
- auto-rebuild VS wip operations?
- global 'timezone' setting (only in TOMLs currently)
- investigate the case where a mulch-secrets.db was erased (0 bytes) during an update + restart
- a Sync() is now done during save(), as for VM database, let's see if it helps
- use os.Rename to atomically save the file (extend to other databases)
- allow to export a port to a different number? (only possible for @PUBLIC now)
- not a quick change, VMPort is not ready for that
- can be a bit tricky to understand for the user? (can be tempted to use "real" source port instead)
- syscall.Exec alternative in cmd/mulch/topics/ssh.go for win compat?
- keep mulch/mulch.env in backup? (with secrets?)
- better error message in case of a double creation?
- vm info: show ports?
- mulch client port forwarding (via SSH)?
- README: add image for vm migration?
- output seems mixed when multiple migrations are targeting the same hosts?
- rewrite README, with "public release" in mind?
- explore vm migration possible edge case:
// migration of active VM ok → local vm deletion → an existing
// "lower" inactive VM is activated (unharmful error if source was active?)
- rights: limit SSH to a specific VM? (check a fake vm=… header in the SSH proxy?)
- rights: logs may expose sensitive data (which ones?)
- rights: add predefined right groups for common usages?
- add "variables" to TOML files? (like $author or $USER in VM name or URL, to create "generic" TOML files)
- provide a whereis feature / add "official" scripts (like wtf_is_my_vm.sh) to the client?
- proxy-chain: provide a way to clean old childs? (ex: proxy_chain_child_url have changed)
- proxy / proxy-chain request stats?
- check domains validity on VM create, PS: domain name validation is HARD :(
- add comments to backups (and other objects?)
- test "pre-allocated" backup disks on backup duration for "big VMs"?
- fix completion when using a non-default (-c) config file (see barry: __barry_get_config)
- on redefine, show what's instantaneous and what's needing a rebuild? a reboot?
- allow to globally disable auto-rebuild?
- allow a cron-style syntax for auto_rebuild
- allow to set a manual rebuild ("at" style?)
- like restore_backup, allow to restore from a VM (ex: autorebuild from prod)
- backup available at "prepare" stage during a restore? / "meta" informations for restore? (ex: gitlab version)
- must be available BEFORE the backup even exists (ex: rebuild)
- purge old secrets (caution with long time disconnected peers)
- even better "secret list" command (closer to the usual FS experience)
- investigate Let's Encrypt throttling issues
- add "ipv4 only" option for seed downloads?
Long term:
- find a way to secure ssh agent forwarding? (so other users can't use the socket in the VM)
- allow a centralized secret share (useful with many hosts)
- "hidden multi-server support"? (with a "proxy mulchd" routing requests to the correct real mulchd)
- full async API?
- write API public documentation
- continue tests on snapshots (see branch "snapshots")
- use vm_prefix for disks/seeds/… to allow multiple mulchd on the same machine (or use different storages?)
- investigate why we seem to lose contact with (some) VMs when killing/restarting libvirtd
- (dhcp/dnsmasq? ebtables? mulch-network restart?)
- libvirtd watchdog + alert (ex: timeout in VMStateDatabase?)
- remove libvirt? (direct use of QEMU/KVM) - warning: network
- create a nice and shiny website for Mulch project
- have a look at ansible? (for sample scripts)
- shortcut for "do action" (ex: mulch open xyz) with completion?
- add 'env' to mulchd.toml? (overridden by VM's env directive)
- check for missing response.Body.Close() (or things like that)
- check for io.Reader bad usages ("they must record the number of bytes read into the buffer, reslice the buffer, process that data, and only then, consult the error." https://dave.cheney.net/2019/09/05/dont-force-allocations-on-the-callers-of-your-api)
- change HTTP code from 200 to 4XX in controllers when things fails (ex: req.Stream.Failure)
- in client, make the non-200 error message silent in stream mode?
- check connection and access rights
- differentiate "rights issues" from "libvirtd not running" issues
- check 'nwfilter-dumpxml clean-traffic' / libvirt-daemon-config-nwfilter during install?
- mulchd should be able to run with a remote libvirtd (StoragePath disallow this, currently)
- support passphrase for Mulch SSH key?
- move SSH Keys and MulchSuperUser to VM level?
- challenge based auth?