Export logs from tpot to SIEM

[vasokolov]

Hello everyone!
I am a newbee in ELK and docker, so i hope somebody will help me to find the way.
I have been install a tpot in my infrastructure, it was hive installation.
So it looks like everything works fine. But now i want to send events from tpot sensors (i mean honeypots logs) to my SIEM system.
I know that every honeypot save all his logs in ~tpotce/data in his folder.
But, as i see, all of this logs going to logstash, so maybe the simplest way for exporting logs on external siem is reconfiguring logstash with some file? but i have no idea how to do it. Also, i think that i don't want to replace a "target point" for logstash, because it will remove data from kibana, i want to add a second, so can i send data from logstash for 2 ways?

[vasokolov]

hi again! so, if you want to export honeypots data to another destination, such as SIEM, you can use logstash container from tpot instance.
First step you should add a syslog output plugin to logstash, (command for this action already include in logstash Dockerfile, but you need to uncomment this).
After that you should build you own image of logstash from this Dockerfile and use this image for logstash on tpot (replace logstash image path in main docker-compose.yml)
And finally, in main docker-compose.yml add a logstash configuration file to a logstash container via volumes of contaier.

I hope this short answer will help somebody with similar situation. Sorry for my english, this is not my mother tongue=)

[akajhon]

Helped me a lot @vasokolov ! Thanks!

[vasokolov]

so, finally i meet a new trouble with logstash output syslog plugin. It does not work very well. logstash-plugins/logstash-output-syslog#51 (comment)
and this

I preffer to use http logstash output plugin, which are already included in TPOTCE logstash instance.
If you want to do this, u should put something like this inside your logstash config. Finaly, you will get a structured similar json events from TPOT inside your SIEM.
http { url => 'http://ip:port/input' # input is only in my case! http_method => post retry_non_idempotent => true format => json }

[Sagarshar06]

Do I have to add environment variables for forwarding syslog to remote ip?

[vasokolov]

sorry, but i don't get what u mean. if u want to forward some event to another destination via rsyslog, or syslog ng, you dont need to add some variables to environment. but it may depence of your personal case. maybe u will find a full answer in rsyslog documentation

[Sagarshar06]

Apologies for not elaborating it. Here is what I want to do! I want to send all the honeytrap logs to my SIEM so that I can make SIEM use cases for monitoring the events. In order to achieve this should I forward all the container logs to Syslog or just the TPOT's syslog?
I tried sending TPOT's syslog to SIEM and It was successful, but I am not getting the event which is required to make use case. The only logs I am getting is related to tpot service whether it is failed or started here is reference log "tpot.service: Scheduled restart job, restart counter is at 4315."

[vasokolov]

if u want catch only honeypot logs, u can go for my way with using logstash. this is work fine, and this is strucrureg log format (JSON), so it is very nice for parsing events, making a rules and cases for SIEM.
If u want to see a honeypot platform log (OS log) and control system.d services, like tpot.service or docker service or anything else, u should connect it on to SIEM like typical linux OS.

[Sagarshar06]

Can you elaborate your way like how did you achieved this what changes did you made in the logstash configuration? Get Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ From: vasokolov ***@***.***> Sent: Monday, August 12, 2024 1:04:04 PM To: telekom-security/tpotce ***@***.***> Cc: Sagar Sharma ***@***.***>; Comment ***@***.***> Subject: Re: [telekom-security/tpotce] Export logs from tpot to SIEM (Discussion #1617) CAUTION: This email originated from outside of Milliman. Do not click links or open attachments unless you recognize the sender and know the content is safe. if u want catch only honeypot logs, u can go for my way with using logstash. this is work fine, and this is strucrureg log format (JSON), so it is very nice for parsing events, making a rules and cases for SIEM. If u want to see a honeypot platform log (OS log) and control system.d services, like tpot.service or docker service or anything else, u should connect it on to SIEM like typical linux OS. — Reply to this email directly, view it on GitHub<#1617 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BEK3DEM5KAE333JHMQNUQQ3ZRBQOZAVCNFSM6AAAAABKW4CALSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTAMZRGA4DEMA>. You are receiving this because you commented.Message ID: ***@***.***>

________________________________ Milliman values and protects your personal information. You can find information on how we use your personal information and how to exercise your rights here<https://us.milliman.com/en/global-privacy-policy>.

[dmille6]

i have done something similar. i'm not sure its the best approach, but it does work.. its currently more a proof of concept.. but it is effective.

Tpot Hive --> ElasticSearch --> OpenCTI --> <Stix/Taxi Feed> --> SIEM

I modified my tpot hive http_input.conf to send the data to the Hive install AND a separate elasticsearch instance.. i didn't want to mess with the hive elasticsearch instance.. so I just send the data to another instance while keeping the hive intact.

#http_input.conf file:

# Output section
output {
  elasticsearch {. #hive elasticserach instance
    hosts => ["elasticsearch:9200"]
    # With templates now being legacy we need to set the daily index with its template manually. Otherwise a new index might be created with differents settings configured through Kibana.
    index => "logstash-%{+YYYY.MM.dd}"
    template => "/etc/logstash/tpot-template.json"
    template_overwrite => "true"
  }

  elasticsearch { #seperate instance 
    hosts => ["http://<URL>:9200"]
    index => "logstash-hive-1"
    user => "<username>"
    password => "<password>"
    ssl => false
    action => "create"	# used for es data streams
    #template => "/etc/logstash/tpot-template.json"
    #template_overwrite => "true"
  }

}

I use a python script run once an hour that queries ES and pulls the tpot data I want and dumps it into OpenCTI: https://github.com/OpenCTI-Platform/opencti

OpenCTI has an option to create a STIX/TAXII feed based on criteria.. so I have a feed.. and this feed is pulled in by various infrastructure. i wish the OpenCTI API was a little more documented.. but there is a good community behind the project, and its a great open source project.

its been very effective and works well.

i hope this makes sense.

[dmille6]

OpenCTI has "connectors" and i'd love to write a connector that pulls in tpot data directly.. i just haven't had time to play with that aspect of the project yet.

[akajhon]

Hello, @dmille6 !

I took a similar approach for shipping the logs to a SIEM.... Modifying the http_input.conf file seems to be the best approach. I Wrote a little script for pulling the data from the TPOT Hive ES and sending them directly via syslog.

Also, loved your approach for creating feeds from the TPOT data via OpenCTI. Would like to recommend the 'GreedyBear' project for this: GreedyBear

Hope it helps!

[dmille6]

thanks a lot, i'll take a look at greedybear.