Create Mordor Datasets¶
You can create your own mordor datasets like the json files available in this repo. The mordor style to do this is by exporting data from a kafka broker and writinng it to a JSON file while executing the simulated test.
In consumer mode, Kafkacat reads messages from a topic and prints them to standard output (stdout). You can also redirect it to a file (i.e. JSON) This means that you can save all the data collected right before you start a simulated test from a Kafka broker. You can stop the consumption when you are done performing the simulated test. You can just grab the logs from this repo and re-play them as if they were being ingested in real-time.
Kafka Broker : A distributed publish-subscribe messaging system that is designed to be fast, scalable, fault-tolerant, and durable (Installed by HELK).
Kafkacat : A generic non-JVM producer and consumer for Apache Kafka >=0.8, think of it as a netcat for Kafka.
Install Kafkacat following the instructions from the official Kafkacat repo
If you are using a debian-based system, make sure you install the latest Kafkacat deb package.
You can also install it from source following the Quick Build instructions.
Consume data being produced from a kafka broker with the following flags:
-b: Kafka Broker
-t: Topic in the Kafka Broker to consume the data from
-C: Consumer mode
-o: Offset to start consuming from (i.e. end)
$ kafkacat -b <HELK IP>:9092 -t winlogbeat -C -o end > empire_dcsync_$(date +%F%H%M%S).json
That’s it! You now can share that dataset with the community!