After some changes in my life; I took a few years away from hands on “hacktivity” to work on other parts of my life that needed some attention. Recently, I tossed around the idea of getting back into technical work, which is how this blog has come about. While getting back into the swing of things on a regular basis, one thing has made my life abundantly easier compared to “the ol’ days”. That’s ChatGPT.
Tl;DR ChatGPT can do helpful things like generate malicious keys and certificate files, lint / correct syntax errors in scripts and commands, and much more.
I don’t want this to be a cliched post about how GenAI / LLMs will change the world and bring the end to programming / technical work. I’m also certain I’m a bit late to the ballgame for writing about this. However, I find it useful to talk about these types of use cases to help my own understanding and hopefully, it’ll help yours as well!
Fortunately for me, a lot in the security world hasn’t changed too much since I last stepped foot in it at this level. However, my recollection of how to dot i’s and cross t’s certainly has become rusty. Fortunately, I still know what I need after a few years off; albeit, from a high level perspective. In the past while shaking off the technical rust, I’d often search Stack Overflow and random forums for the parameters I needed for sets of command, configurations, or just to brush up on how a specific programming language worked. Now, all of that is a magnitude easier.
It has especially made my Security Research and Penetration Testing easier with the utilities and guidance it provides, which is what we’ll dive into today. This list will outline some helpful and / or surprising features of ChatGPT that accelerated my testing.
#1 – Malicious Certificate and Key Generation
I know I know, you probably have certificate and key generation memorized so this is probably not useful for you. Unlike you, this would have taken me 15-30 minutes of research and testing at minimum to get exactly the results I would have liked. ChatGPT did this in less than 10 seconds. Not only did it tell me exactly what I needed to do, it also decided to do the most surprising thing, which was doing it for me!
While I was performing some testing on my local Jenkins instance, I was thinking through different ways of injecting payloads that weren’t the typical HTTP Header/Body/Inputs that I’m used to. As part of this, I discovered the Jenkins Credential Plugin had an ability to upload a P12 Cert aka a PKCS#12 Cert. I thought to myself, the web input sanitization and output encoding is great for all of the typical stuff. Maybe it’s worth trying something nefarious using the certificate file itself. Lo and behold, ChatGPT.
Prompt:
Generate a p12 file with malicious inputs such as XSS and XXE and SQL injection in the different field names.
Response:
The Jenkins UI rendered the fields in the preview prior to upload! It worked like a charm!
However, it did not lead to any vulnerabilities 😦
Overall, this is one of the most surprising technical outputs I’ve had in many, many years and the first time I’d audibly said “Wow” while sitting at my desk.
#2 — URL / Parameter Encoding
For me, long gone are the days of running encodeURIComponent() in the Chrome Console or using https://www.urlencoder.org/ to quickly decipher JSON or turn my payloads into something consumable by the server. I’m certain there are even more efficient ways than these two that you all know. But going forward, for me it’ll be with the help of ChatGPT.
Example of Decoding:
Example of Encoding:
This isn’t just for URL and Parameter encoding / decoding, I used it for Base64, Hexadecimal, and other forms as well.
This to me is an example of how ChatGPT will become like a smart phone. Slowly but surely building a suite of capabilities that make going to specific software utilities and solutions obsolete. There’s a meme that goes around with the below picture that says “All of these things are now in your smartphone!” — I think ChatGPT and similar GenAI / LLM services will head in this direction. Services like my dear, dear, https://www.urlencoder.org/ will likely lose their place in my toolbox that they once had and sit as a memory or screenshot on my desktop similar to the below.
#3 — Alternative Payload Suggestions
I love manually testing forms and inputs. It’s a guilty pleasure of mine and I feel it still yields very positive results. Yes yes, I know, I should have a list of payloads, bypasses, and malicious inputs surrounded by automation at my finger tips to test automatically. A far away state compared to the manual inputting and modifying of every field and parameter for each form on a webpage that I do today. To me, there’s something more intimate to testing manually and understanding exactly the inputs and outputs of an application versus running a suite of payloads and automated detections and tests. That being said, sometimes I get stuck and I need some alternative ways of addressing a problem, and that’s where ChatGPT also shines.
While testing a Stored XSS I had found, there was a pesky CORS (Cross-Origin-Resource-Policy) implementation that prevented me from exfiltrating data. Knowing it won’t help me necessarily defeat CORS itself, I decided to ask a slightly different question to open the idea box. The first 8 or so suggestions didn’t really help, i.e logging the cookies / sessions to the console. See below — however
The 10th item surprised me and helped remind me that exfiltration via an <img> tag is a very common thing to do and the response from ChatGPT was helpful for me to get past the mental road block that was stopping me from achieving the outcome I needed.
Again all of these things may seem basic and trivial to some of you, but it provides just the right amount of boost to my rough edges to help get me across the line.
#4 — Linting / Syntax Correction
Similar to the first one, I’m sure you have fancy IDEs that automagically add in the correct parenthesis and brackets. For me, I like to keep payloads and scripts that are very tactical in nature in Notepad++ with minimal linting and syntax correction. This helps when I am intentionally trying to leave curly braces and the like unbalanced for injection purposes. However, when I go down the rabbit hole late at night and I’m too lazy to sort out which bit is missing; ChatGPT to the rescue!
In this scenario I was attempting to use Javascript to load content from a payload (test.html) that I had made onto a system through a XSS vulnerability I had discovered. Somewhere along the way I dropped a parenthesis and it was 2:30AM so there was little chance I was going to use my brain power to do this myself!
All I did was send in the payload and ask it to fix it nicely and viola! A working and improved code snippet 🙂
Generating (and improving) code is a staple of ChatGPT for quick and scrappy functionality that saves a ton of time. This one falls towards the end since it is likely the most common and understood use case of them all!
#5 — Sanity Checking / Devil’s Advocate
Now that I’ve built somewhat of a routine around the technical work I’ve been diving back into, something I have regularly are check-in’s with ChatGPT to discuss ideas and general direction that I am heading in with a project. Yet another job that ChatGPT will take! Rubber Ducky Debugging! Rubber Ducks across the world are shuddering at the thought! In all fairness, Rubber Ducks are awesome, but they don’t respond and you have to fill the in the void with your own opinion and takes when discussing topics outside of walking through code line by line.
With that being said, quite often I start with a wild thesis or speculation and look to disprove them quickly by failing fast. Asking ChatGPT to play devil’s advocate or to be critical of your thought process helps debate the merits of your ideas and act as a sounding board. The amount of poorly rendered web responses and error logs / exceptions I’ve thrown at ChatGPT it to speculate about with me is growing by the day. It usually starts with the idea that there is an underlying flaw in the system based on a random hunch or small lead that I want to quickly debate. Most recently I went on a path to find some poor input sanitization in Jenkins with only partial success (findings for a later blog post). So I asked ChatGPT, am I being reasonable in my (at the time) initial findings?
Example Query:
Response
Of course! ChatGPT wants to be nice and suggested a myriad of reasons. Most of which were already in line with my thinking, which is either awesome for ChatGPT or embarrassing for me. I’ll let you all decide. Obviously I don’t want my own opinions, which is why I always ask for a contrarian opinion if it confirms my beliefs. With the simple phrase “Play devil’s advocate” the response with a dose of reality kicks in. Obviously things like Jetty are robustly tested and it is unlikely to have a significant vulnerability that I can find that others can’t.
All of that being said — even with it playing devil’s advocate it came to the same conclusion I did, it’s worth poking around a bit more to see what may exist that others haven’t already found 🙂
With all of that being said, I hope you found this article useful for your security research, pen testing, or just overall technical work that may lie ahead. If there are tricks or other useful things ChatGPT does for you, let me know in the comments below!