-
Notifications
You must be signed in to change notification settings - Fork 14
/
Copy pathNEWS
970 lines (679 loc) · 28.9 KB
/
NEWS
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
v0.29.2
=======
Adds the ability to ban fields based on regular expression. Also improves
the replace transformer that only supported metadata fields, now it also
supports the data field.
Adds new data type support to cast transformer. Now you will be able to
transform between byte-arrays (blobs) and strings.
v0.28.0
=======
Release date: 2024-09-27
Adds the ability to ban content with a regular expression. Also bumps some,
but not all modules.
Fixes an issue where transformers were not visible when running with -show.
PS: There were some github action issues that caused some minor history
issues which can be seen in the git log as we re-did the tags of v0.28.x
v0.27.2
=======
Release date: 2024-03-18
Very minor release to improve debugging by adding timestamps to
metric-related messages. Also a few library bumps for security advisories.
v0.27.1
=======
Release date: 2023-12-01
Minor, second attempt at fixing the ban transformer. Should no longer
cause out of bounds errors. Hopefully.
v0.27.0
=======
Release date: 2023-11-22
Two minor, but important bug fixes:
- Fixes an issue with out-of-bounds checking on the ban transformer.
- Make the SNMP trap sender iterate over all metrics in a container, not
just the first.
Other than that, some more relevant documentation and examples.
v0.26.0
=======
Release date: 2023-10-12
At a glance: Improvements to the SNMP sender and security fixes.
This addresses:
- CVE-2023-37475 (avro issues)
- CVE-2023-39325 (HTTP/2 rapid reset)
The SNMP trap sender fixes included make the sender more usable. It allows
manually configuring snmpTrapOID among other things, needed to make the
sender actually send valid snmpTraps.
v0.25.1
=======
Release date: 2023-09-25
Allow Dictsplit to split on the top-object.
Previously Skogul could split::
{
"metrics": [
{
"metadata": { },
"timestamp": { ... },
"data": {
"foo": {
"eth0": { "key": 123 },
"eth1": { "key": 567 }
}
}
}
]
}
Into::
{
"metrics": [
{
"timestamp": { ... },
"metadata": { "name": "eth0" },
"data": { "key": 123 }
},
{
"timestamp": { ... },
"metadata": { "name": "eth1" },
"data": { "key": 567 }
}
]
}
but always needed a top key bellow "data" to refer to. Now you can also
split the following original::
{
"metrics": [
{
"metadata": { },
"timestamp": { ... },
"data": {
"eth0": { "key": 123 },
"eth1": { "key": 567 }
}
}
]
}
To do this, simply omit the Field option (or leave it as an empty array).
E.g.::
"transformers": {
"type": "dictsplit",
"metadataname": "name"
}
As before, "Metadataname" will be the metadata key where the dictionary
index is stored.
Also, some printf-debugging was accidentally included in v0.25.0, this is
removed.
v0.25.0
=======
Release date: 2023-08-21
Skogul, now with more blobs.
The blob encoder and "parser" introduces a way of shuffeling arbitrary data
through Skogul without actually parsing it. By using the blob "parser",
Skogul will simply add the input data to a generic "data" key as a byte
array. This can then be forwarded and handled with any regular Skogul
mechanism. The regular JSON encoding will actually base64-encode byte
arrays, so you can forward it just fine like that.
To unpack the data, you can use the blob encoder, which should return the
data to the original unparsed state. Since you may have introduced a batch
sender and put multiple metrics into a container, the blob encoder has an
optional delimiter which will be injected in between metrics.
The TCP sender expands upon the net-sender. Technically, the net-sender can
do TCP just fine, but it is extremely inefficient at scale, because it will
open a new TCP connection for each container sent, instead of re-using it.
The TCP sender tries to address this and has been tested. It is still not
perfect in error-handling, so it should not be used if you need a guarantee
that messages are delivered, but it is actually reasonably performant.
Putting this together you can receive syslog messages on a UDP port and
forward it with the TCP sender without either side knowing there was
something like Skogul in between, and thus making sure rsyslogd doesn't get
backed up and blocks due to a slow receiving end and full buffers. NOT THAT
THAT EVER HAPPENS.
docs/examples/misc/tcp.json includes an example of two receivers using the
same handler, the blob encoder and tcp sender.
v0.24.0
=======
Release date: 2023-08-08
Adds two new, somewhat experimental modules: an SNMP trap sender and
rabbitmq sender and receiver. Both are expected to get some changes after
we get them tested, but of course any feedback is welcome.
v0.23.0
=======
Release date: 2023-06-06
Bump Juniper telemetry protocolbuffers from Junos 21.3R1 to 23.2R1.
Adds a small fix to the influx sender, preventing it from sending
empty tag strings to Influx. Fixes #294.
Adds support for renaming metadata and data fields using the "rename"
key to the metadata/data transformer. Example::
"transformers": {
"ok": {
"type": "data",
"rename": [
{ "source": "old_key_name", "destination": "new_key_name" }
]
}
v0.22.0
=======
Release date: 2023-05-09
Add JSON unmarshalling to SQL receiver, add unflatten to metadata
transformer as well, and allow banning with ban transformer by path.
v0.21.1
=======
Release date: 2023-02-27
Add SIGHUP-handling to file-sender.
Add JSON-casting to cast transformer.
Upgrade to Go 1.20, and bump various libs past security lol.
v0.21.0
=======
Release date: 2023-02-10
Add NATS sender and receiver (thanks Niklas Holmstedt!), add support for
"match any"/presence in switch-transformer and fix a minor issue where
blank string could never match.
v0.20.11
========
Release date: 2023-01-05
Releases v0.20.0 to v0.20.10 are all build-system/CI/CD-related tweaks and
basically irrelevant to end users.
The following is the release notes for v0.20.0:
This is a small, but important release that adds support for loadable
plugins, and also the LineFileAdvanced file reader that allows log
rotation-based file reading (e.g.: Move a file, send sigup, then parse it).
The plugin support is very much experimental, see docs/plugins.rst for
discussion. There's also an example-plugin at
https://github.com/telenornms/skogul-plugin-example which demonstrates how
to build an out-of-tree plugin.
v0.20.10
========
Release date: 2023-01-05
Releases v0.20.0 to v0.20.10 are all build-system/CI/CD-related tweaks and
basically irrelevant to end users. See v0.20.0 for actual notes.
v0.20.0
=======
Release date: 2023-01-05
This is a small, but important release that adds support for loadable
plugins, and also the LineFileAdvanced file reader that allows log
rotation-based file reading (e.g.: Move a file, send sigup, then parse it).
The plugin support is very much experimental, see docs/plugins.rst for
discussion. There's also an example-plugin at
https://github.com/telenornms/skogul-plugin-example which demonstrates how
to build an out-of-tree plugin.
v0.19.1
=======
Release date: 2022-11-15
This is an "emergency release" of sorts, that REMOVES SQLITE SUPPORT.
We need to nuke this because it introduces what turned out to be
unacceptable dependencies which can't be (easily) met on, e.g., CentOS 7.
This means Skogul 0.19.0 RPMS wont work on CentOS/RHEL 7, while Skogul
0.19.1 should.
Sorry for the inconvenience. We will re-visit SQLite when a more
appropriate library can be used.
v0.19.0
=======
Release date: 2022-11-14
Somewhat large release, but:
- Drone is on the fringe, so manual release. GH actions incoming.
- New SQLite sender
- New *Experimental* prometheus parser
- New unflatten transformer
- Cast transformer: Add casting of IPs to integers
Minor:
- Several minor chores from Håkon Solbjørg that increases consistency and
correctness, with minor negative user-impact.
- Splunk: Send metadata as fields to spluk HEC
- Deperecate "Path" and use "File" instead for consistency (deprecation,
old configs will keep working)
- Add KeepOriginal flag to the Data transformer to
- Many log cleanups
All in all, this is a larger release than we'd like, but also largely
isolated features.
Since this will be a manual release, it might deviate SLIGHTLY from the
other releases, but should be mostly the same.
v0.18.0
=======
Release date: 2022-09-20
Adds USP protocol buffer parser and AVRO encoder/parser time support, adds
worksarounds for Juniper -Inf transceivers, adds dummy parser.
USP
---
The USP parser will parse a protocol buffer-encoded USP message contained
in a record, and use the payload as JSON. This might change if hardware
surfaces that use USP differently.
AVRO
----
The AVRO encoder/parser now handles time fields, using a minor workaround.
Shouldn't require any changes for users.
Juniper
-------
We now apply various hacks to fix #218, where juniper telemetry reports
-Infinity, which is impossible to encode as JSON. Intead, we silently
convert it to -40, horrifying any mathematicians in the room.
Dummy parser
------------
The dummy parser is meant as a developer-tool where we can save un-parsed
bytes to disk, thereby allowing us to acquire test data for development.
Not useful in any production environment.
v0.17.0
=======
Release date: 2022-08-24
In short:
- AVRO and GOB encoding and parsing
- SQL receiver
- Encoder-support in file sender
- Docker images!
Docker images - first release, so let's see how well this works :D
SQL receiver will periodically poll an SQL database for data, it is a bit
immature, but it should be solid. You write a query, the query needs a
field named 'time' (select foo as TIME ...) for the timestamp, and you also
select which columns are metadata - the rest will be data.
While GOB could theoretically improve performance, it's not possible in
practice because GOB isn't schemaless, but embeds a schema in the initial
packet. This means that GOB data depends on the receiver having the initial
packet, and there is no convenient way to achieve that except
re-initializing the encoder every time. The end result is that gob is no
better or worse then JSON in regards to performance, but, unlike JSON, it
can support every data type that Go supports, fully, without losing
accuracy.
The AVRO format is still missing support for timestamps, see #247, but
that's being worked on.
To use AVRO, you need to specify a schema. One is provided in
docs/examples/avro, which is a very generic schema. You can write a more
specific one as well, for your specific data, but so far there is no
support for time stamps. To be able to actually USE Avro today, you need to
add a transformer on the handler to add a timestamp.
v0.16.0
=======
Release date: 2022-05-28
TL;DR:
- Kafka sender and receiver
- Default configuration file moved to /etc/skogul/conf.d in systemd units
- Enricher transformer MUST be used together with enricher updater sender
- "Wholefile" receiver will read a file and treat it as a single container
- Much needed docs/examples cleanup
- Dummy timestamp transformer
If you are using an unaltered systemd unit file and our RPM packages, you
will have to move your configuration from /etc/skogul/default.json into
/etc/skogul/conf.d/ - the file name can be whatever, as long as it ends in
.json - Skogul will load every configuration file present.
This better allows dynamic configuration through, e.g., Ansible and is
what we've been using for a while.
The Kafka sender has one little quirk: Since Kafka allows batching messages
natively, it was decided that it is better for overall performance to
encode each metric in a container as an individual message, instead of
encoding the entire container as one message. This will better exploit the
nature of Kafka and make it easier to use the data, hopefully.
The enricher also went through a major simplification: It no longer has a
native loader function, and instead you have to configure an enricher
updater sender. The configuration becomes slightly more cumbersome, but the
end result is a much more elegant solution.
v0.15.2
=======
Release date: 2022-05-09
Adds plumbing for modular encoding, but more importantly, an enrichment
updater sender.
The sender "enrichmentupdater" and the corresponding changes to the
enrichment transformer will allow on-line updates to the enrichment
database using regular Skogul pipelines. You can load a JSON file at
start-up with an initial database of enrichment, then update that database
on the fly over an HTTP interface for example, if you set up a
receiver-pipeline that ends in an enrichment updater.
Both the enrichment updater and the enrichment transformer is STILL
experimental, but I'm feeling good about them, and they will be a major
addition to medium to large-scale Skogul deployments.
PS: I botched the NEWS in v0.15.1 so let's go lucky v0.15.2.
v0.15.0
=======
Release date: 2022-04-10
Add an experimental enrichment transformer, and a slightly less
experimental switch sender.
The enrichment transformer WILL change, but needs testing, so it is
included. It allows an operator to provide a JSON file to enrich metrics
based on metadata fields, e.g.: Add customer name to interface stats, or
add if description. To use it, see docs/examples/tester_to_stdout_enrich.json ,
you will need a JSON file that has a list of metadata fields to match, and
the data field will be the fields added *to metadata* (yes, this is
confusing, this is one of the reasons it's considered experimental).
The switch sender is simpler, but also probably a bit immature, but testing
is required. It takes a list of conditions and checks if a metric(!)
matches this, and if it matches a complete set, data will be sent to the
associated sender. Use case: Dupe sender sends data to your own influxdb,
and to a switch sender, the switch sender then selectively streams
customer-specific data to a customer.
v0.14.3
=======
Release date: 2022-04-01
Introduce float flattening to the cast transformer, which allows flattening
floating point numbers into "non-exponential" string representations. E.g.:
For serial numbers, instead of having them encoded as 3.141519e+X when sent
to influx as tags, they'll be encoded as 3141519.
Introduces much better request logging as well for the HTTP receiver.
v0.14.2
=======
Release date: 2022-03-22
Introduce IgnorePartialErrors in handlers to let a handler modify a
container by removing invalid metrics instead of dropping the entire
container.
Also introduces several small but important log-quality fixes.
v0.14.1
=======
Release date: 2022-03-22
Add Cast transformer, which allows specifying a shallow schema for input.
It can cast data and metadatafields to strings, integers and 64-bit floats.
It currently ignores missing fields, and for fields already of the correct
value, it "just works" with no added costs. For fields needing conversion
it will somewhat cumbersomely go through fmt.Sscanf(fmt.Sprintf(...),...),
which is somewhat costly, but also ensures that it can convert anything
that fmt knows how to handle reasonably well.
It is meant to use as a "we can't fix the source data", not as a "we don't
want to fix the source provider".
v0.14.0
=======
Release date: 2022-03-22
Updates juniper telemetry interface, including a change of protobuf
compiler. Tested on various routers.
Greatly improves error messages, specially for protobuf, this is the
beginning of a major revamp to use wrapped errors, which will hopefully
both reduce log spam, and make the log messages more useful for real life
trouble shooting. For protobuf, this means actually useful errors when
marshaling fails, including what router and the underlying errors. Next up:
Workaround for -Infinity.
Also adds http stats, courtesy of eiriktaa <[email protected]>.
Also switches rpm builds from CentOS 8 which no longer works at all, to
Rocky Linux 8. This should have no impact on end users, regardless of
platform.
PS: This is a verbatim copy of the entire v0.13.x debacle, since it failed
to build the release. There are 0 code changes between 0.13.0 and 0.14.0,
and this is just to get the actual NEWS section fixed and a clean start.
v0.13.2
=======
Release date: 2022-03-22
Only build fixes since 0.13.0 failed to check on our CI pipeline. See
v0.13.0 for actual news.
v0.13.1
=======
Release date: 2022-03-22
Only build fixes since 0.13.0 failed to check on our CI pipeline. See
v0.13.0 for actual news.
v0.13.0
=======
Release date: 2022-03-22
Updates juniper telemetry interface, including a change of protobuf
compiler. Tested on various routers.
Greatly improves error messages, specially for protobuf, this is the
beginning of a major revamp to use wrapped errors, which will hopefully
both reduce log spam, and make the log messages more useful for real life
trouble shooting. For protobuf, this means actually useful errors when
marshaling fails, including what router and the underlying errors. Next up:
Workaround for -Infinity.
Also adds http stats, courtesy of eiriktaa <[email protected]>.
v0.12.1
=======
Release date: 2021-12-10
Fixes a bug in the stats receiver where it would not run transformers before
sending data.
v0.12.0
=======
Release date: 2021-12-10
Adds a Stats receiver for shipping internal skogul stats. This is a bit more
detailed than the Stats _sender_, and will be further extended in the future.
For now there are stats for the UDP receiver, protobuf parser and HTTP sender.
Furthermore, in the future we will try to get rid of more log messages about
errors and rather use stats for that.
Fixes the Structured Data parser to support more formats, and tackle all
formats with the same efficiency and priority.
Adds a concept of skogul.Identity to identify a skogul module by name. The
name is fetched from the configuration, and is typically the key used to
define a module.
This release also adds CopyFromData, a transformer to copy a field from data
to metadata without removing it. The ExtractFromData transformer is marked as
obsolete and will be removed in the future. To achieve the same effect, use
CopyFromData togheter with a Remove transformer.
v0.11.0
=======
Release date: 2021-12-02
The Batch sender now has a concept of a burning metrics if the sender is
too slow. If a "Burner" sender is configured, the batch sender will
redirect metrics to the burner if the regular sender would otherwise block,
e.g.: If the backlog is too big. This is recommended for setups where data
is duplicated to multiple backends, such as to a staging and test
environment, where you don't want the staging and test environment to slow
down data replication to the production environment.
Additionally, the UDP Read buffer size can now be configured for the UDP
receiver, which can be very useful for bursty telemetry data.
On a side note, shout out to Lasse Haugen for inspiring the DictSplitter
included in v0.10.9.
v0.10.9
=======
Release date: 2021-12-01
Skogul now figures out if the configuration path is a file or a directory,
and will work with either using the same options (-f or -d). We will
probably deprecate one of the flags at some point in the future.
Add a new DictSplit transformer.
Improved language in documentation.
Make the null-sender configure itself automatically.
The Net sender now checks for required parameters.
When running tests, try to find a new port if the selected one was already in use.
v0.10.8
=======
Release date: 2021-11-15
Fixes a bug that would crash skogul if it failed to convert
a protocol buffer packet into the internal data format of
skogul. The underlying problem was an interesting value
for a numerical metric, which there are now tests added for.
v0.10.7
=======
Release date: 2021-05-07
Fixes the systemd(1) unit file so that it supports being enabled
and automagically started on boot. We all WantedBy this.
v0.10.6
=======
Release date: 2021-04-27
Adds even better support for escaped characters in the influx parser,
specifically if the values had previously been escaped.
v0.10.5
=======
Release date: 2021-04-27
Supports parsing escaped hex, decimal and unicode character sequences
in the influxdb parser. Previously this would not have the expected behaviour.
v0.10.4
=======
Release date: 2020-11-12
Adds support for specifying a metadata field for Source in the Splunk sender
or setting it as a constant.
Adds experimental support for specifying what loglevel to log a failure as
in the UDP receiver.
v0.10.3
=======
Release date: 2020-11-06
Adds support for renewing the Client ID in the MQTT client upon reconnects,
as reconnecting (too quickly) with the same client ID might make the broker
forcibly close the connection after having reconnected. Enable this option with
RenewClientID: true.
v0.10.2
=======
Release date: 2020-10-26
Makes UDP receiver packet size configurable, instead of being hard-coded to 9000.
This was the first contribution from someone outside our organisation, thanks a lot
to @moogacs for this feature!
v0.10.1
=======
Release date: 2020-10-16
Fixes a memory leak in the Structured Data parser. No other parts of
skogul were affected by this bug.
Other, small fixes:
- Remove a silly debug log line
- Inform the user if they supplied an invalid log level
v0.10.0
=======
Release date: 2020-10-14
This is a big one! The changes are mainly new and opt-in features, so things
should work "just as before".
Features:
- New sender: splunk. Can be used to send to Splunk HEC.
- New parser: Structured Data (RFC5424). Only the data, not a syslog.
- New transformer: parse. Can be used to apply a parser to a
specific field in the data.
- new feature: TLS Client Certificate support in HTTP sender and receivers.
- new feature: TLS Client Certificate SAN DNSName verification.
Fixes:
- UDP receiver has been buffed with more robust handling of requests as well
as some more tests.
v0.9.0
======
Release date: 2020-09-25
Promotes parsers to modules, making the configurable. To keep simplicity,
parsers can be "auto-instantiated" without having to create a "parsers"
key in the configuration file. To configure a parser, a "parsers" section
has to be added and the appropriate configurations made.
The auto-instantiation feature has been added to the debug sender and the
template transformer as well, to reduce the amount of boilerplate required.
Breaking change: Previously, not supplying a parser in a handler was allowed,
and would default to use the json parser for skogul containers. This is no
longer the case, and a parser always has to be defined.
v0.8.1
======
Release date: 2020-09-22
Adds a parser for MNR data and adds some more logging to the file sender.
v0.8.0
======
Release date: 2020-09-04
Adds a file sender which supports writing to file.
Fixes the RPM build spec so that rpm will not overwrite the default config
file upon upgrading skogul.
v0.7.2
======
Release date: 2020-07-30
Fixes a bug in the influx parser so that it now supports '=' (equals sign)
in tag values without dropping the rest of the tags.
v0.7.1
======
Release date: 2020-07-03
Fixes a bug in the influx receiver so that it now supports '=' (equals sign)
in tag values. This is useful for command lines where arguments might be
-foo=bar.
v0.7.0
======
Release date: 2020-06-18
Configuration can now be read from multiple files by using the new `-d`
option instead of `-f` to refer to a configuration directory instead of a
configuration file.
All configuration files need to end with `.json`, and all are read. When
all configuration files are read, any reference is reviewed. This means
that you can have a set of standard senders in a configuration file and
refer to them from an other.
v0.6.2
======
Release date: 2020-04-03
Fixes a bug introduced in the influx-sender in v0.6.1 which would send any integers
as quoted string values, as in `key="123i"` instead of the expected `key=123i`.
Fixes a bug in the influx-receiver which would skip the final byte in a float
when parsing it.
v0.6.1
======
Release date: 2020-04-03
Fixes a bug in the influx-sender which didn't send integers to influx as <int>i,
which is what InfluxDB expects. Since this is a breaking change and requires the
measurement fields to be dropped/updated/altered, there's a feature flag added:
"ConvertIntToFloat". Enabling this will convert ALL numbers sent through that
sender to FLOATS. Don't do this unless you know you need this. The default value
is OFF.
v0.6.0
======
Release date: 2020-04-02
Introduces a parses for the InfluxDB line protocol. The parser is to be considered
in beta/testing. Activate it by configuring a handler with the "influx" parser.
v0.5.5
======
Release date: 2020-03-03
Fixes a bug with the build pipeline which would reuse a previously generated
skogul binary when archiving releases. The binary in question did not contain
the build flag for setting version number, so now we clean the directory before
building a release to make sure to rebuild it with the proper flags.
v0.5.4
======
Release date: 2020-03-03
Fixes a bug where a previous bugfix for generating builds with version numbers
wasn't in master and therefore wasn't present for future builds.
v0.5.3
======
Release date: 2020-03-03
Fixes a bug with -version which wouldn't work for prebuilt binaries/packages.
v0.5.2
======
Release date: 2020-03-02
Fixes a bug with the install RPMs which would error during install
and uninstall. The RPMs are now built on a system running yum, which
will make sure build dependencies are correctly satisfied. Also reworks the
build system using Make, and introduces a -version flag.
Refactors the internal Handler API to expose Parse, Transform and Send
(also TransformAndSend which transforms a container and then sends it,
and Handle which parses, transforms and sends a container). This should
make the API easier to understand and makes sure relevant procedures (
like Validate()) are run before e.g. Send()-ing things off.
Specific changes:
- Fixes a bug with the RPM building which would make it error
during install and uninstall (reinstall). The package gets
installed correctly, but not uninstalled. This bug has to be
manually worked a round if any previous skogul RPM is
installed, by manually uninstalling the version using rpm.
(rpm -e --noscripts skogul*)
- Inverts the influxdb-sender field validation to white-list instead of
black-list data types, which should fix #119.
v0.5.1
======
Release date: 2020-01-30
Add custom flattening separator, and a magic 'drop' separator to
remove any intermediate keys. Also adds some benchmarks for custom
parsers as well as cleaning up some package names for test files.
Specific changes:
- Support for custom flattening separator as well as dropping intermediate keys.
v0.5.0
======
Release date: 2020-01-22
Skogul 0.5.0 is a small release, but should make life much easier for
InfluxDB-users.
Specific changes:
- InfluxDB sender now detects invalid metrics instead of sending "garbage"
to InfluxDB that fails - it will send what it can.
- InfluxDB sender has much better logging
- Split transformer now correctly transforms remainder of metrics in a
container after encountering one it can't handle.
- RPM packages and systemd unit files now agree upon default config file
path.
v0.4.1
======
Release date: 2020-01-06
Skogul 0.4.1 brings support for exposing MQTT topics in the metadata. Yup, that's it.
v0.4.0
======
Release date: 2019-12-04
Skogul 0.4.0 brings MQTT configuration up to speed with other receivers and senders.
A new transformer to set the timestamp is introduced, as well as a custom JSON parser
for use where the incoming JSON format does not conform to the one of Skogul.
Furthermore, new versions now ship with prebuilt RPMs as well as prebuilt archives.
We also run automated tests on all example configuration in ./docs/examples!
Specific changes:
- JSON syntax errors offsets correctly with tabs
- Support building RPMs and build RPMs for new releases
- Bring MQTT configuration up to speed (configuration updates required!)
- MQTT wildcard support (+ and #)
- A custom JSON parser, for use when the JSON is not already on Skogul format
- Timestamp transformer, used to set the timestamp of a metric from data
v0.3.0
======
Release date: 2019-11-18
Skogul 0.3.0 contains several changes aimed at project management, some to
improve core functionality, and some module improvements. It's not a huge
change since 0.2.0, and that's how we like it. As per usual, we aim for
high quality releases.
Specific changes:
- Use of CI and automated building of release tar-balls
- Cleanup and refresh of documentation
- There's now a single unified way of handling modules that dramatically
reduces code duplication in core.
- Modules can now provide documentation for nested data structures, which
is correctly exposed in manual pages.
- A conditional transformer (#71) was added that provides a mechanism to
only execute a transformer if certain conditions are present.
- The "extract" feature of the metadata transformer no longer creates empty
metadata keys, and will remove the original to avoid duplicating
information.
- The logrus receiver replaces the log-receiver, allowing better control of
log data. More work in this area is to be expected.
- The HTTP receiver can now apply different authorization rules for
different paths.
The latter constitutes a bit of a change in configuration.