Summary: Valve was using IP-based whitelisting to restrict access to their Dota 2
partner.dota2.com endpoint, but was using an untrustworthy mechanism for determining what the visitor IP address was.
I will try to give a more basic step-by-step account of trying things, so if you can guess what’s coming, you’re probably right and you can give this a skip!
Finding a target
Being the hardened, but only partially employed, internet thug that I am, I was cruising certificate transparency logs for various web properties, looking for opportunities for mischief. I had watched some BTS King’s Cup that morning (Dota 2 esports tournament), so they (dota2.com) were on my mind.
As an aside, CT logs are a great recon tool to discover what companies are doing - you can find lots of semi-private facilities, sometimes while they are in-progress of being set up - and it comes with date information!
One entry definitely caught my attention:
Partner? Curious. Valve have international operational partners for the game (Nexon in South Korea and Japan and Perfect World in China) that run the Dota 2 game for them in their respective regions, so maybe it was that. Dota 2 has a huge esports scene as well, so my first thought was that it referred to their tournament production partners (like PGL).
Finding an angle
The HTTP response from the domain itself was pretty juicy (my own IP altered):
This response is wonderful. It both reveals the method of authentication (IP address) and also reflects its server-perceived value back to me (the 220.127.116.11). It’s also a dead giveaway that it’s probably a fun target, if its users have their own Valve contacts!
Having previously worked in web hosting for some years, I was familiar with the fact that there are some common security pitfalls with figuring out what the visitor’s IP address is. A great thing to try exploit.
When you have two peers that directly establish an HTTP session over the internet (like a web browser and a web server) this is straightforward. You can extract the IP address from the layer 3 information and it is pretty much guaranteed to be truthful, barring some weird setups.
When you have more than two peers that speak HTTP in the chain (proxies, CDNs), this same means of getting the IP address is unhelpful, since it is just the IP of the previous peer (at least, in the case of layer 7 proxies). So, to get the actual visitor’s IP address, each peer needs to cooperate to pass on that piece of information through a different mechanism. This is usually done via the
Sometimes, in certain common misconfigurations (see pitfalls above), these servers are not setup in a way that guarantees the integrity of the XFF header.
What should happen (✔): The first (or last, depending on how you look at it) peer that receives an accurate layer 3 IP address SHOULD ALWAYS set the value of the XFF header, even if it was already present in the request, and then each remaining peer trusts that value.
What sometimes happens (✘): if the header is already present in the request (e.g. set by the browser), the servers won’t overwrite it and will just trust it.
I would try to test whether the latter scenario was true in order to get past this screen.
The good ‘ol tale of trusting user input
What IP address to try spoof though? We don’t know what the permitted addresses are, but having worked on software and infrastructure a bit myself, I find that
127.0.0.1 usually makes an appearance for a couple of reasons: a) people use it in development and b) sometimes the webapp needs to make loopback requests. In the end, even if
127.0.0.1 wasn’t there, we could always just slowly brute force all of the v4 prefixes in Valve’s AS and hope they don’t notice.
My first attempt was just to straight up send the XFF header in an HTTP request. This didn’t work.
$ curl --compressed -i -H 'X-Forwarded-For: 127.0.0.1' https://partner.dota2.com/ HTTP/1.1 500 Internal Server Error Server: nginx Content-Type: text/html Vary: Accept-Encoding Content-Encoding: gzip X-Varnish: 975514878 Content-Length: 100 Date: Mon, 18 Sep 2017 07:06:38 GMT Connection: keep-alive Cannot authenticate.<br>Provide info to your Valve rep:<br>Address: 18.104.22.168
(we must use –compressed because the server sends gzip responses no matter what ¯\(ツ)/¯).
The unchanged reflected address reveals that spoofing this header doesn’t work :(.
I decided to do some more reconnaissance to find other avenues. We already know the server is nginx (and Varnish is also in the chain, since we see a header from that too), but where is it hosted? Are they using some kind of CDN or WAF or anything?
$ dig +short -x $(dig +short partner.dota2.com) a104-74-27-241.deploy.static.akamaitechnologies.com.
Yup. Akamai. I had never used any Akamai product, but as a CDN for HTTP, it could not be that different to Cloudflare, who have their own (notably, trustworthy) headers you can rely on for the visitor IP address.
So, the next step is to Google for ‘akamai x-forwarded-for’, which takes us to this forum thread where we read about a header called
Whatever, I didn’t read the thread after seeing the header name, but I gave that a shot:
$ curl --compressed -i -H "True-Client-IP: 127.0.0.1" "https://partner.dota2.com" HTTP/1.1 200 OK Server: nginx Content-Type: text/html Vary: Accept-Encoding Content-Encoding: gzip X-Varnish: 845634917 797960397 Content-Length: 69 Date: Mon, 18 Sep 2017 07:12:33 GMT Connection: keep-alive Specify a partner on the URL, with ?partner=<partner>.%
Super! Since we get a different response, we have confirmation that:
True-Client-IPis spoofable. For some unknown reason, Akamai decided to implement this header in a totally untrustworthy way. In my research after-the-fact, I discovered that they know this and wanted to change its behavior all the way back in 2015.
partner.dota2.comweb application is using the
True-Client-IPheader, in some form, and is assuming that it is trustworthy.
What even is this partner thing
OK, so the previous response was demanding us to provide a
partner query parameter.
It is said that security through obscurity is no security at all, but I am pretty sure that if Valve didn’t show its hand regarding using IP authentication, and also not revealed the query parameters of the following page, I would have given up long ago.
As discussed previously, partner could have a few meanings. Tournament production partner was the first thing that came to mind, so I tried aimlessly for a little while using strings like
';-- without getting anywhere:
$ curl --compressed -i -H "True-Client-IP: 127.0.0.1" "https://partner.dota2.com/?partner=pgl" HTTP/1.1 500 Internal Server Error Server: nginx Content-Type: text/html Vary: Accept-Encoding Content-Encoding: gzip X-Varnish: 966386014 Content-Length: 93 Date: Mon, 18 Sep 2017 07:22:09 GMT Connection: keep-alive Set-Cookie: partner=pgl; path=/ Cannot authenticate.<br>Provide info to your Valve rep:<br>Address: 127.0.0.1
A few minutes later, Googling for ‘partner.dota2.com partner’ led me to this forum thread on dev.dota2.com where ‘GermanViet’ points out, in a totally unrelated context, the possibility that partner could mean Nexon or Perfect World. Thanks random forum dude!
nexon and we got a real response!
$ curl --compressed -i -H "True-Client-IP: 127.0.0.1" "https://partner.dota2.com/?partner=nexon" HTTP/1.1 200 OK Server: nginx Content-Type: text/html; charset=UTF-8 Expires: Mon, 26 Jul 1997 05:00:00 GMT Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Pragma: no-cache Vary: Accept-Encoding Content-Encoding: gzip X-Varnish: 972676446 Content-Length: 2517 Date: Mon, 18 Sep 2017 07:24:21 GMT Connection: keep-alive Set-Cookie: partner=nexon; path=/ <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> <html> <head> <link type="image/x-icon" href="https://partner.dota2.com//public/favicon.ico" rel="shortcut icon"/> --- rest of response is redacted
And we’re in
I won’t say too much about what was on the site itself because a) I don’t think its important and b) there’s something there but I didn’t fully understand what it was.
Something that gave me a laugh was that because we used
127.0.0.1 as the IP address to spoof, the page seemed to render debugging information about the page render itself, which revealed some sensitive information about the host it ran on, what other servers it depended on etc.
Otherwise, from that page you could go on to login via a Steam account (which I did) which took you to other restricted facilities.
Curiously when I got the Steam 2FA email for a new login location, the originating IP of the Steam authentication operation was
gw.valvesoftware.com / 22.214.171.124. I wonder if this could have implications for launching other attacks - SSRF anyone?
Finally, I emailed Valve and took my dog for walkies.
- 2017-09-18: Wake up at 5am, stay in bed watching King’s Cup until 10am, find this issue, email firstname.lastname@example.org
- +12 hours: Valve acknowledge receipt
- +4 hours: Valve have fixed the issue and the header is no longer spoofable