Intro to Threat Detection with YARA

For troubleshooting/help with this lab, please join our discord server and I’ll be glad to help!

 

Skills you will obtain:

• Identifying basic file IoCs
• Creating YARA Rules
• YARA
• yarGen
• Arya (YARA rule testing)

I highly encourage you to put this on your resume under your project/home lab section. A couple quick ideas that you can go off of:

• Created YARA detection rules by manual collection of simulated malicious document IoCs.
• Utilized yarGen to generate YARA detection rules for simulated malicious documents.
• Tested created/generated YARA detection rules with Arya, a tool that creates pseudo-malicious files.

Obviously, you’ll want to tweak these bullets as you see fit. I encourage discussion within the Discord channel for the lab on this.

Next, I’ve listed the repositories for all three tools you’re going to need for this lab. While YARA can be run on Windows, we’re going to do all of this in Linux. I prefer Kali Linux, but feel free to run with whatever distro you want.

Repo for YARA:
github.com/VirusTotal/yara

Just type yara in the command line, if you don’t have it, it should prompt you to download it. Select ‘Y’ for yes on both questions it prompts you with.

Repo for yarGen:
github.com/Neo23x0/yarGen

Run the following commands from your shell:

mkdir ~/tools && cd ~/tools
git clone https://github.com/Neo23x0/yarGen.git

Repo for Arya:
github.com/claroty/arya

For this one, make sure you’re in your ~/tools directory still.

git clone https://github.com/claroty/arya.git

One last thing:
PLEASE, PLEASE, PLEASE don’t open this file and view it, just follow the instructions or you’ll ruin the fun of the lab.

git clone https://github.com/Jayteaare/labscript.git

**This will create a folder called labscript, with a file by the name of lab.sh (this will set up the lab environment, nothing crazy though).

Lab:
To begin, navigate to wherever you put the final file you downloaded (the .sh file), then type the following command:

***DO NOT RUN THIS COMMAND AS ROOT***

bash lab.sh.

Navigate to your Desktop. You should now see a new file. For this lab, we’ll say an end user noticed a suspicious document on their desktop. Now, we’re going to run some analysis on this to make some indicators of compromise (IoCs).

First, right click the file and go to properties. The initial things we can get for IoCs are the name of the file, the extension, file path, and size in bytes.

So far here is our IoC list:

• File name: kenobi.txt
• File path: /home/%USER%/Desktop
• Size: 24 bytes

Now, we can run the strings tool on it. Strings is a tool that can gather collections of characters that produce ‘human readable’, well.. strings? Most of the time you’re going to get a ton of garbage characters when running this. So to get into a good habit, we’re going to pipe the text produced by this tool into a new .txt file.

Realistically, with this being a text file, you could just cat it out or put it into a text editor. With real malicious files though, especially executables, you want to use strings. We’ll get in more depth in the upcoming lab of malware analysis. Shameless self-plug 😉 Keep your eyes peeled in the future.

Run the following command:

strings kenobi.txt > kenobistrings.txt

After, run cat on the new strings .txt file. Now we can see what text the file contained:

• “Hello there! star{warz}”

A bit gamified, but we’re doing this to get a solid foundation, and a general understanding of what YARA is capable of. Bear with me.

So now our IoC list is:

• File name: kenobi.txt
• File path: /home/%USER%/Desktop/kenobi.txt
• Size: 24 bytes
• Strings: “Hello there! star{warz}”

Now, let’s get the MD5 hash for the file by using the following command:

md5sum kenobi.txt

Typically, I like to grab the MD5, SHA-1, and SHA-256 hashes of any file I’m dealing with. This will come into play in the upcoming Malware Analysis lab as well, just something to keep in mind for the future. Go read about hash collisions if you aren’t aware of that term already.

For the future, below you can find the commands to calculate  both the SHA-1 and SHA-256 hashes of a file:

sha1sum <file>

sha256sum <file

Let’s do one last IoC list update before we continue:

• File name: kenobi.txt
• File path: /home/%USER%/Desktop/kenobi.txt
• Size: 24 bytes
• Strings: “Hello there! star{warz}”
• MD5 Hash: f1bc52b1c4da8b1d9dbe44bf41697d9d

AGAIN, I want to stress to you that this is a very basic glimpse into what YARA is really capable of. If this lab interests you, I suggest doing further research into the tool yourself.

Now, let’s start making a YARA rule. Afterwards we can use YARA to hunt for any additional files that meet our IoC list.

Run the following command:

nano starwars.yara

There are a couple of pretty basic concepts to understand with YARA rules:

• You have to name the rule.
• You can use metadata.
• You can use strings.
• You have to use conditions.

We can do a deep dive into the specifics of these sections in the Discord, just trust me for now. You can see the rule I wrote in the screenshot below.

Can you guess what’s next? Yeah, we’re going to use the rule.

Use the following command:

sudo yara -m -s -r starwars.yara ~/ 2>/dev/null

(-m feeds the YARA rule file, -s shows the strings on which YARA detected, -r searches recursively)

You’re going to get hits for a few files. Namely, kenobistrings.txt, kenobi.txt, starwars.yara, and lab.sh (please ignore this one, for obvious reasons). We are already aware of all these files. However, there is one new file: grievous.txt. It’s located within the /home/%USER%/ directory.

Let’s navigate to the ~/ folder now and check out this additional file. In the spirit of good habits, we can run through the process of collecting IoCs for this file. Attempt to walk through the basic set of IoC collection steps we did on the original kenobi.txt file. If you forgot, I have the list below.

• File name: grievous.txt
• File path: /home/%USER%/grievous.txt
• Size: 38 bytes
• Strings: “General Kenobi.. star{warz} order{66}”
• MD5 Hash: e10edced67b747f31ce544a5ec39b4eb

We can now update our rule with the additional strings and run it again.. just in case, you know?

After you’ve updated your YARA rule (you can see mine above), run the following command:

sudo yara -m -s -r starwars.yara /tmp 2>/dev/null

At this point I think you understand the general concept of YARA and its uses. Once you’ve collected IoCs for a malicious file, you can then use a YARA rule to scan an entire system to look for any other instances of them. This can help a ton when you find an interesting reference to a file within strings, etc.

Now, let’s check out yarGen and how it can be used to generate rules automatically.

Run the following command:

cd ~/tools/yarGen/malwarez && ls

As you can see, there is a file called deathstar.txt. Let’s collect our simple IoC list for this file before we continue:

• File name: deathstar.txt
• File path: /home/%USER%/tools/yarGen/malwarez/deathstar.txt
• Size: 90 bytes
• Strings: “VGhpcyBpcyBub3QgdGhlIGRyb2lkIHlvdSBhcmUgbG9va2luZyBmb3Iu O66. DV. LS. #%%%^storm##tropper”
• MD5 Hash: 046d58458878c1cbafa660dfa497e86c

Now, run the following command to place you inside the yarGen folder itself:

cd .. && ls

Run the following commands:

pip install -r requirements.txt (if you don’t have pip, just enter pip, it will prompt you for downloading just like YARA)
python yarGen.py -u

Note: python yarGen.py -h will show you a selection of command arguments you can use with yarGen.

Run the following command:

python yarGen.py -m ./malwarez/ -o starwarz.yara

Note: If your system is killing the process while it’s running, you need to give your VM more RAM.

After running this command, it will create a file named starwarz.yara to your current directory. Go ahead and open the file in a text editor, such as nano or vim.

As you can see, it utilizes a couple of additional file attributes we haven’t covered yet. I’m going to leave it to you to dig additionally into these attributes for YARA, as we will cover them in the next step of this YARA series.

Now let’s go ahead and run this rule with YARA to prove that it works:

sudo yara -m -s -r starwarz.yara ./

Well guys, there’s your quick run down of YARA and yarGen usage. These tools have a lot to offer. If this interests you, definitely continue to expand your knowledge.

Drum roll please… Are you ready for your culminating event? I’m going to give you some strings and hex code so you can write your own YARA rule. No other help. Once it’s complete you are going to utilize a tool named Arya to test it.

First, run this command within the Arya folder located in ~/tools/arya:

sudo apt update && sudo apt install yara cmake && pip install -r requirements.txt

Strings:

• yoda
• grievous
• kenobi
• deathstar

Hex:

• 63 3a 5c 77 69 6e 64 6f 77 73 5c 73 79 73 74 65 6d 33 32 5c 62 6c 61 68 2e 65 78 65
• 62 6c 61 68 2e 64 6c 6c

Other file attributes:

• 491 bytes
• uint16(0) == 0x5a4d

I’m sure you’ve noticed I included some additional attributes we definitely explicitly go over. It is completely up to you what you use for your YARA rule. I encourage you to play around with Arya and string identification within your rule (i.e. fullword ascii).

Once you’ve made your YARA rule, run the following command:

python3 ~/tools/arya/src/arya.py -i <path to your rule> -o youdidit.exe

If all goes well, Arya will return that your rule has been ‘Triggered’. If it doesn’t, don’t give up! Keep toying around with different ideas on how to get this to work. Arya isn’t a perfect tool, but the use case is something like: you know some of the data of a malware file (such as strings) from some article you’ve read, but you don’t have a legit copy of the malware sample to test your rule on; this is where it comes into play. While not a super necessary tool, it could prove to be useful in some use cases.

That concludes our ‘novice level’ introduction to YARA, yarGen, and Arya. I hope you all enjoyed it! Feel free to DM me some constructive criticism on Discord, any questions, or some of your YARA rules!

-Jayteaare