Dead canaries in your network

When an adversary is inside your network, the faster you can detect and remove the intrusion the better.  Even if you don't have a "network" per se – even if you are running a pure zero-trust environment – detecting an attacker at work early will give you the upper hand.

Even with sophisticated EDR products in the mix, criminals can often introduce malware to an environment to gain a foothold in a way that isn't detected.  Introduction of malware and establishment of a foothold is critical to criminal operations and so today's criminal gangs spend a great deal of time and resources using tradecraft and techniques to bypass the detective and preventative controls running on user workstations.  Even with a really good set of tools in the hands of an experienced defence teams, there is a good chance of criminals starting their attack chain without being caught. 

Most breaches start with compromised credentials or a phishing attack, and end with data being stolen for sale or publication, or a ransom demand.  The top discovery method for breaches (accounting for more than 50% of discoveries), according to the Verizon 2022 Data Breach Investigations Report, is now Actor Disclosure – i.e., the intrusion isn't detected until the criminal posts a ransom note or advertises your data for sale. 

An adversary within your environment is at an inherent disadvantage compared to your systems administrators.  When that adversary is a criminal, they need to learn all about your internal operations quickly, and from a position of disadvantage – they have to conduct a lot of internal reconnaissance to understand the technologies in use internally, the information repositories you have and the systems in operation.  Your systems administrators have a great understanding of your internal workings; a criminal adversary is – at least initially – fumbling around in the dark.

You can spend a lot of money on controls which track access to systems and resources, and aim to flag unusual events or access patterns that might indicate a compromise in progress.  All of this is really valuable and helps improve detection rates when it comes to intrusions.  You should definitely do this work! 

Comprehensive detective monitoring is, however, difficult – you need resources to implement, manage and triage monitoring and alerting solutions.  You'll need to be careful to manage the telemetry you receive from these controls, and need to spend time understanding false positives, and trying to minimise false negatives for any alerting you have in place.  Tracking down false positives and eliminating them in day-to-day operations can be time-consuming for teams and – worse – if your teams get too used to false positives they can become desensitised to the alerts they see on a regular basis.  Few organisations, though, make much use of a (relatively) simple, effective and attacker-unfriendly technique to support detection.

Enter the canary: the Lego bricks on the floor that your attackers will tread on whilst they're trying to find the proverbial light switch.  

What's a canary?  It's something you leave lying around which is just too tempting for an adversary to avoid investigation, or which an adversary doesn't know to avoid when they're trying to move around, grab your data and compromise your assets.

Canaries can be great.  Since they have absolutely no legitimate operational use, triggering of one of your canary alerts is generally a clear, urgent indicator that you have an intruder in the network – you don't have BAU false positives to filter out.

Maintenance is a breeze – typically, these are fire-and-forget within your network, and you don't have to look after them once you've put them in place.  Whenever your network changes, you can pop some more canaries in, or take some out – and still have another way to trip up criminals who are trying to navigate your internal networks.

When we compromise a workstation or gain access to a cloud account during an attack simulation, we rarely end up achieving our objectives by a single direct attack.  Typically, we need to move laterally through a network or escalate our privileges before we can access the systems or data that we want to, or perform our attacker-like objectives (like causing a denial of service, or simulating ransomware).  Our general approach to privilege escalation and lateral movement is fairly standard – identify the resources we can see from our position, determine whether any of the data we have access to can help us gain access to other accounts or systems that might be helpful, look for any exploitable vulnerabilities exposed to us, and then plan our next step towards our objectives.  

In practice, this can mean all sorts of things.  We'll look through file shares we can see from a user workstation, searching for system configuration files which contain hardcoded passwords, text files with configurations or user passwords and system documentation.  We'll browse Active Directory and look for likely candidates for password guessing attacks and members of privileged groups.  We'll read documentation on intranet sites like SharePoint (and search these for passwords too).  We'll take a look at public information sources like GitHub repositories and see if there's anything interesting stored there that we might be able to use internally.  We'll scour the local filesystem for information stored in configuration files at build time.

When we find stuff that looks interesting, we make a note and then – and this is where you can lay down your Lego bricks – we have to check whether it works and experiment with it in order to move laterally.  And we'll generally give a preference to the easiest, most-direct and fastest route we can find towards objectives.

Let's imagine we find a file server which looks like it contains user home directories, and we want to look at the contents.  To do this, we need to: 

• Identify the file server on the network – we might identify the service via port scanning, or via a tempting DNS name / Active Directory entry (fileserver01, home-directories, etc.).
• Connect to the file server to query contents via the protocol (e.g., SMB)
• Identify interesting file names and retrieve contents of these, or search the contents of all files on the fileserver for keywords

If the "fileserver" we are looking at is full of fake-but-looks-real data, and has no place within your normal business operations, the fact that somebody connects to the file server at all is a huge red flag.  Searching and reading file contents is even greater cause for concern, and any interaction with this canary server gives you excellent confidence that there is something wrong so that you can act. 

You don't need to confine yourself to placing whole honeypot systems within your network – you can embed canaries all over the place.  You can create configuration files for fake applications which contain hardcoded database credentials and pop them on your file servers, or in your documentation.  When somebody uses those credentials to authenticate to a database account, you have another clear, high-quality indicator that somebody is up to no good.  You can deliberately put AWS access keys – for accounts with no rights – onto user workstations, or into documentation. If anybody uses these keys, you know they've been accessed.  You can create accounts within Active Directory with bad passwords whose sole purpose is to provide an alert if anybody uses those credentials.  

A good systems administration team, given a couple of hours, could come up with a huge list of canaries you could put into your network.  

There are a few things to bear in mind.  Of course, you don't want your canaries to make things worse for you when an intrusion happens, so you need to be careful to make sure that the use of a canary doesn't inadvertently assist your attacker.  If you're going to make Active Directory accounts to be compromised, you need to prevent these from being used to gain extra privileges.  If you are going to make canary access tokens for AWS, you need to make sure these accounts don't have access to any resources or permissions.  

You also need to be careful not to document your canaries in your online information repositories – so an adversary who is gathering information can't identify them easily.  It also bears repeating that you shouldn't make your honeypots or canaries obvious by calling them things like "honeypot01" or "canary-server" in DNS.  (Yes, we have seen people do this.)  

Most of the time, a criminal won't know you have canaries.  When they don't know you have canaries, the chances are that they will check every piece of information they find to validate its usefulness – and if you have plenty of canaries, this will mean lots of high-quality alerting information which is obviously Known Bad.  

If a criminal knows you use canaries, they won't know what is real and what is a trap when they infiltrate you.  In the worst-case scenario for you, this means they will be very slow and careful in their actions and might be sufficiently lucky to avoid any of your triggers.  But this knowledge will absolutely slow them down significantly – and in many cases it may mean that you're not worth targeting when softer targets are available.

We often say that the playing field between attackers and defenders isn't level, and that attackers often have the advantage.  Think about laying traps for your attackers to even the score.  You won't catch all of them, but you and your systems administrators can make it very difficult for even the most determined adversaries to move laterally undetected with a bit of imagination.