Further Securing Pi-Hole Access from Internet
Over the course of the last year (year and a half) I've been working on methods to securely access my Pi-Hole admin interface from the internet without needing a VPN. I've explained this thought process several time on the r/pihole subreddit, but I'll put it here as well just for a bit of background.
If you go onto shodan right now and search Pi-Hole you'll come across 445 IP addresses where the Pi-Hole admin interface is exposed to the internet.
Despite the moderators of the r/pihole sub reddit removing questions about making Pi-Hole available online, and the developer team informing users that no part of Pi-Hole should ever be exposed, people still do it. There used to be countless questions on r/pihole of people asking "how do I access Pi-Hole when I'm on the go"; to which the default answer is always "just use a vpn."
I've gotten into multiple arguments with the Pi-Hole mods/devs about this in the past. A VPN is a good solution but it is not right for everyone. Take my unique situation for instance; when I'm at work I can't use a VPN on my work computer to access my home network and whitelist a website for my wife or kids. I have to access the web interface, because of this I need a way to do it securely.
The Pi-Hole devs have been rather obstinate about this, adamantly refusing to properly secure Pi-Hole and instead suggesting that there's no need to secure it because it shouldn't be on the internet in the first place. As most of us learned years ago but clearly the devs are still catching up on; things need to be built secure. Leaving a vulnerable device on the internal network and saying "oh it's fine because its internal" is a very dangerous prospect, one that has caused major issues for massive companies like Target in the past.
I've also been around the security sector long enough to know that people are going to do what they want anyways. User is a four letter word after all. Try as we might if someone wants to open Pi-Hole to the internet they will and refusing to help them do it securely isn't going to stop it. Instead all the refusal does is create a larger vulnerability as now the dev's purposely unsecured webserver is publicly exposed on the internet.
One of the first issues I wanted to address was using HTTPS instead of straight HTTP to access the admin interface. One of those security by design issues I mentioned earlier is the lack of HTTPS. If I'm only using Pi-Hole internal this isn't a major issue; its not good, but its not a major issue. It does become a major issue if you're accessing Pi-Hole from anywhere on the public internet. HTTP is unencrypted so if you try to login to your Pi-Hole from Starbucks then potentially anyone could now have your Pi-Hole password.
My first post addressed setting up Docker, Portianer, IP Tables, and a reverse proxy to enable HTTPS access to your Pi-Hole.
What I learned after writing that first post is that Pi-Hole as another security by design flaw. The Admin interface does not log access attempts. This means that trying to set up something like Fail2Ban to prevent brute forcing the admin password is not possible without completely modifying the Pi-Hole code. I brought this up to the Pi-Hole devs and was again greeted with "just don't put Pi-Hole on the internet".
For my second post I wanted to address this issue of brute forcing the admin password so I went with a zero trust platform from Cloudflare (Cloudflare Access). Following this guide results in visitors having to authenticate through the Cloudflare proxy and be on my approved list before they can access my Pi-Hole (at least that's what I thought). I still standby the guidance about UFW that I put there as that will work but the IPTables section was lacking.
As I continued learning and building I added a significant amount of additional services, many of which were in Dockers. Managing the UFW rules for services not in Docker and the IPTables for the Docker became quite difficult and I honestly messed it up with Docker IPTables.
Ok, so it wasn't actually an attack, but it did point out a flaw in my set up. But before I get to that I'll detail out how we came to this attack.
About 2 years ago I bought Google Wi-Fi because my Wife was tired of the issues I was running into learning MicroTik and wanted something that just worked. While Google Wi-Fi works (when Google isn't pushing out broken firmware) it is very barebones. One of the core and basic features it's missing is Logs. I eventually replaced Google Wi-Fi with Orbi which has logs.
After setting up Netgear Orbis I started offloading and analyzing the Logs in Splunk (I'll do another post on that later). When I started reviewing those logs I noticed that my NGINX reverse proxy was being hammered by bots just going straight to my public IP. This was my indication that my Docker IPTables were not set up correctly.
I did check through my NGINX logs to see if there was any actual compromise but I didn't see anything that was concerning. To the best of my knowledge the bots simply went to my <publicIP>:80 and <publicIP>:443 but since there was nothing there they moved on.
This did make me curious about if I could bypass CloudFlare access completely.
When I was analyzing the logs with my coworker we started to hypothesize about how to manipulate my proxy to allow access to these servers without going through Cloudflare. Looking at how NGINX works we know that it listens for a specific URL and then forwards that to the appropriate server based on the config file, which is why simply going to my public IP didn't work for the bots.
What we originally theorized was that we could use BURP Suite to manipulate an HTTP header and add a specific URL to the header. By going to the IP directly and then adding the URL to the header we would not DNS search and therefore not go through the Cloudflare proxy bypassing the Access requirement.
It was actually more simple than that. We didn't even need to use BURP to add a URL to the header. All we needed to do was add the URL and my public IP to the windows host file. As soon as we did that going to one of my subdomains went right to my proxy and straight to the server.
Not good. (I still use strong passwords on my servers so it wasn't as bad).
Breaking down the attack
This attack was actually really simple and could have been devastating if used maliciously. Lets take a quick look at how Cloudflare Proxy/Access works
With normal traffic your system will query an authoritative DNS server to get the IP, but with Cloudflare being my authoritative DNS it will return its own IP as the server. Then Cloudflare will proxy the traffic to my actual source.
This is why if you put my domain in a DNS lookup tool it will return Cloudflare as the IP address.
But Let's not forget what the actual first step for DNS is. Before your computer reaches out to a DNS recursive resolver or to the name server it will check its own host file. If the URL is in the computers host file then it won't ask Cloudflare for the IP and Cloudflare will therefore not provide the IP address of the Access Proxy. (ie. the computer will connect directly to my proxy).
This means that Cloudflare is not checking my ACLs and because my IP Tables were set up wrong they were allowing access from anyone.
We tested this and were able to drop directly to my Pi-Hole admin login page without needing to authenticate through Cloudflare Access.
I want to start off by saying this isn't an issue with Cloudflare. Cloudflare is upfront in their documentation for Access and Argo that you should set your firewall rules to only allow Cloudflare IPs.
Since I didn't have a customizable Firewall when I set this up I could only use UFW and IPTables. If you are more familiar with IPTables and how Docker Chains work in IPTables then you might be able to prevent this.
For me, I bought a pfSense router/firewall and installed that inline then configured it to only allow Cloudflare IPs to connect to my network. Afterwards we retested this and were unable to access the network. Port scans also show the port as closed.
Overall this was an interesting test. I started this journey because I wanted to access my Pi-Hole from my external without a VPN and throughout this I've learned a ton.
There's another test I want to do to see if I can still bypass this as I think there might be a way but I'll keep that post secret until after I've had time to prove/disprove this attack.