R1Soft CDP kmod Build Server Vulnerability
Quick background: R1Soft CDP is a continuous data protection (a.k.a incremental backup) solution, that, at least on Linux, requires a closed-source kernel module loaded in order to operate. Because there are too many Linux kernel variations to reasonably pre-build, they provide a build server where you can upload kernel headers and it spits out a .ko kmod for you.
I found this a curious case, not because it is technically compelling (it isn’t), but because a vendor that produces otherwise quality software (the R1Soft CDP product itself) turned out to have made such amateur mistakes with a critical[1] public web service.
Hey, what’s this thing?
The story begins with me trying to get the Linux hcpdriver module (the name of the kmod that R1Soft relies on) to build after updating our servers at Serversaurus following CVE-2017-6074. For one server, the kmod being saved to the server was an ASCII file with the following contents:
Warning: fopen(/mnt/data/kmod/cached/release/4/hcpdriver-cki-2.6.32-673.26.1.lve1.4.22.el6.x86_64.ko) [ function.fopen]: failed to open stream: Permission denied in /var/www/cust_build/cache-module.php on line 57
Warning: fread(): supplied argument is not a valid stream resource in /var/www/cust_build/cache-module.php on line 58
Warning: fclose(): supplied argument is not a valid stream resource in /var/www/cust_build/cache-module.php on line 59
A couple of emails back and forth with support didn’t prove helpful so I went off on my own to investigate why the build server was failing.
The build server is at https://krnlbld.r1soft.com (which, as of 2017-03-06, gets an F on Qualys SSL Labs for reasons including Logjam, FREAK and protocol downgrade attacks) providing a UI where you can upload your kernel headers and select the build parameters:
(Happily, I was able to get my kmod built via this UI. shrug.)
After submitting a tarball through this interface, you would be assigned a build ID (numeric), and directed to subsequent process pages:
/process.php- The target of the web form, acceptingcki_client,version,archanduserfileparameters./buildstatus.php- Waiting/polling page while build is in progress, withrparameter (build ID)/download.php- Page to download the completed kmod, withrparameter (build ID)
(Some pages omitted, but they have similar issues).
After seeing, earlier on, that the application was so fragile that it was not checking possible errors from file reads, it was hard to resist the temptation to tack on a ' on to the end of a query parameter …
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '\'' at line 1
As it turns out, practically every single parameter listed above is vulnerable to SQL injection of some kind. The application appears to strip quotes, but does not address seemingly any other form of attack.
Issues
1. Various SQL Injection in process.php
By altering the version parameter to something that isn’t expected (in this case, prepending some ../../, fishing for some path vulnerabilities), we reveal that the parameter is naively processed to generate the version and version_major columns).
We also reveal the table structure of ksession (and potentially other tables) and generate an opportunity for error-based SQL exploitation.
Other parameters (such as arch) probably suffer similar issues but I did not check.
2. Error-based SQL exploitation in buildstatus.php
https://krnlbld.r1soft.com/buildstatus.php?r=1 or invalid&c=1
Error-based exploitation can be used here to extract information, even though data is not reflected in page output.
3. Error-based SQL exploitation in download.php
https://krnlbld.r1soft.com/download.php?r=350189z&cki_client=
As above.
4. Code injection in download_success.php (unconfirmed)
https://krnlbld.r1soft.com/download_success.php?r=
I believe there is an opportunity here to do some variant of XSS based on prior inputs, but I don’t have a PoC for it.
5. Tainting module builds (unconfirmed)
If we take a look at the PHP error output that started this investigation, we’ll notice that the application might be caching output of kmod builds:
Warning: fopen(/mnt/data/kmod/cached/release/4/hcpdriver-cki-2.6.32-673.26.1.lve1.4.22.el6.x86_64.ko) [ function.fopen]: failed to open stream: Permission denied in /var/www/cust_build/cache-module.php on line 57
Depending how they identify unique kernels (it may simply be the version parameter we saw above, or perhaps it is a hash over the provided tgz), there is an opportunity to “poison the well”, by either using available SQL exploitation techniques, or even uploading kernel headers for future versions that would produce invalid, or even malicious kmods.
A couple of tests showed that it is not hard to modify the kernel headers and for the output to be reflected in the built kmod. However, it’s not clear to me which, if any, headers can be modified to inject new code. One mechanism might be to re-#define a syscall or known function to sneak code in. I suspect it’s possible, but not demonstrated.
Commentary
The lack of care taken for the public build server is kind of alarming.
When the kmod is closed source and not particularly auditable beyond reverse-engineering, you would hope that it would be built and distributed with care.
However, we have some indications that suggest otherwise.
For internal or abandoned facilities it is perhaps understandable that maintenance lapses. However, this server is in the critical path for their flagship product and it looks a bit abandoned.
Timeline
- 2017-06-04: Published
- 2017-03-29: Bumped support ticket with reseller, informed of intent to publicly disclose.
- 2017-03-07: Reseller has reported the issues to R1Soft.
- 2017-03-06: Discovered issues, could not find direct contact line to R1Soft, reported to our licence reseller.
1. Critical because it is spitting out code that is run with kernel privileges against many thousands of servers. Combined with the SSL problems on the site, seems like a great target for TLAs, if you’re the paranoid type you’ve been awake at any time in the last 5 years.