Dark AI tools: How profitable are they in the underground ecosystem?

Threat actors are constantly looking for new ways or paths to achieve their goals, and the use of Artificial Intelligence (AI) is one of these novelties that could drastically change the underground ecosystem. The cybercrime community will see this new technology either as a business model (developers and sellers) or as products to perpetrate their attacks (buyers).

In June 2023, this became reality with the publication of an AI tool that promised escaping ChatGPT boundaries, named WormGPT. This new tool is based on GPT-J-6B, an open-source Large Language Model (LLM). It was presented by its creator “Last[1], who had allegedly been working on it since February 2023, as a “ChatGPT Alternative for blackhat”. Its promotion and commercialization were done through Telegram and underground forums, while being offered for prices that fluctuated between $100 for a month access or $5,000 for private setup.

First WormGPT post by "Last"

Figure 1: First WormGPT post by “Last”

Days after WormGPT’s launch, on July 22nd, a user under the moniker “CanadianKingpin12” announced another AI tool named FraudGPT. This was also marketed through Telegram, underground markets, and forums [2]. Outpost24’s KrakenLabs researchers noticed that the same user advertised three additional AI tools under the names DarkBERT, DarkBARD and DarkGPT. 

As forecasted, users in the underground ecosystem saw AI as a potentially lucrative business and as a result, these new AI tools without boundaries emerged. They have been created for malicious purposes and are being sold as a solution to skip ethical frontiers, elevating social engineering to another level of complexity.

Moreover, as usually happens when new trends emerge, sellers and creators of these new AI dark tools don’t always deliver as promised. Scammers have also proliferated through fake Telegram channels where they promise tools that will never be delivered to the buyers.

The dark artificial intelligence (AI) business

All the tools observed have similar features, such as writing malicious code, finding leaks and vulnerabilities, or creating phishing pages. However, prices vary depending on who is selling the product, and which product is being sold. For instance, Outpost24’s KrakenLabs analysts observed that CanadianKingpin12’s most valuable dark AI tool was DarkBERT, based on it being the most expensive. As shown below, prices between the developer (“darkstux“) and the seller (Last) of WormGPT were also quite different. 

AI TOOLSELLER/DEVELOPERPRICES
WormGPTLast (seller)1 month – 100€1 Year – 550€Private Setup – 5000€
darkstux (dev)1 month – 90$3 months – 230$6 months – 500$12 months – 850$
FraudGPTCanadianKingpin121 month – 90$3 months – 200$6 months – 500$12 months – 700$
DarkBERTCanadianKingpin121 month – 110$3 months – 275$6 months – 650$12 months – 800$Lifetime mode – 1250$
DarkBARDCanadianKingpin121 month – 100$3 months – 250$6 months – 600$12 months – 800$Lifetime mode – 1000$
DarkGPTCanadianKingpin12Lifetime mode – 200$
**Note: the data above, has been retrieve from the main post in forums, Telegram, and markets.

On June 28th, a user under the moniker “Last” announced the launch of WormGPT in the notorious hacker forum Hackforums. Shortly after, another user named “Laste” created another post publicizing WormGPT in ExploitIn, a well-known Russian-speaking hacker forum. In the ExploitIn forum, the user claimed to have been working on this project since February 2023.

WormGPT Group
Membership:Seller: Last, Laste, Updgame, ruiunashackers[1] DeveloperDarkstux
Type:Crime-Syndicate
Category:Group
Motivation:Personal gain
Objective:Financial gain

During July, Last made several updates to their posts, adding information about the model they were using and other significant updates for the project. One notorious post was made on July 15th, when Last said to a forum user that their developer was “thinking of buying the whole project”. This confirmed that they are a group and that Last was just the seller. 

Last suggesting that their developer might buy the whole project

Figure 2: Last suggesting that their developer might buy the whole project.

The day after Last’s post about their developer’s intentions, a new allegedly legit Telegram channel appeared: “t.me/wormgptai”. This account warned about scammers and referred to the “t.me/wormgptdev” Telegram user as a buying contact. However, a few days later, Outpost24’s KrakenLabs analysts found references in a Telegram channel to “t.me/wormgptai” and another user “t.me/darkstux” as the buying contact instead. This new Telegram channel was even mentioned in wormgpt[.]ai, a site created to promote the tool.

Image showing a website advertising WormGPT

Figure 3: Image showing a website advertising WormGPT.

Even though this new legitimate Telegram channel and its admin (Darkstux) had promoted WormGPT and created the web page, on July 21, Last added the following comment to their Telegram channel:

“We don’t have any website and either any other groups in any platform. We only have two “official” threads, one on hackforums and other on exploitin. The rest are resellers or scammers!”

Thus, creating distance between them and the other seemingly legit Telegram channel. It is yet unclear whether they are related or not.

All of this can be summarized with the following timeline:

  • 28/06: A user named Last advertised WormGPT in Hackforums.
  • 14/07: The same user published the same project but in Exploit.In under the name Laste.
  • 15/07: Last explained in Exploit.In that their developer would like to buy the whole project.
  • 16/07: A new Telegram channel appeared (“t.me/wormgptai”) controlled by a moniker dubbed darkstux, likely the developer.
  • 21/07: New Telegram channel was created “t.me/blackhatai” which later was renamed to “t.me/wormgptdevelopment”, controlled by Last.

Oustpost24’s KrakenLabs researchers’ analysis suggests there is a realistic possibility that Last finally sold the project to its developer, who goes by the name Darkstux.As a result, both have their own Telegram channels offering the WormGPT tool, illustrating how profitable AI tools can be for the underground ecosystem.

Schematic view of WormGPT group

Figure 4: Schematic view of WormGPT group

Exploring the dark AI ecosystem

On July 21st, a careless forum user and customer of WormGPT publicly shared their public-facing API that was being used by the AI tool at that moment. Strikingly, this API lacked any form of authentication, allowing anyone with the correct URL to engage with WormGPT in conversation without any cost involved [3]. This carelessness hinted at the fact that the actors behind WormGPT were not taking security as a priority.

This suspicion was further confirmed when Outpost24’s KrakenLabs researchers discovered that Last had been sharing screenshots containing their current login panel’s IP in their main telegram channel. Although redacted, images have been able to be reconstructed to get a partial IP address. That was enough to successfully identify the machine that was hosting said panel. Despite the abrupt closure of the project some days later, it was still up and working. 

Login panel for WormGPT customers

Figure 5: Login panel for WormGPT customers

CanadianKingpin12 is the seller of different illicit services within the underground ecosystem. They have been active in Telegram and underground markets since July 2023, apparently returning from a time off.

CanadianKingpin12
Aliases:CanadianKingpin, Thecashflowcartel, Canadian Kingpin12, theblackrebus, CanadianSmoker12, CanadianSmoker
Type:Criminal
Category:Individual
Motivation:Personal gain
Objective:Financial gain

From the frequent use of “myself” in the publications, it can be inferred that under this moniker there is likely just an individual. As shown below, they are quite well-known and appreciated by the underground community for their good services:

CanadianKingping12 is likely male based on the above
CanadianKingping12 is likely male based on the above

Figure 6: CanadianKingping12 is likely male based on the above.

Since end-July, they have been offering different AI tools, allegedly being just a service provider and not the developer, as they said within the post in which they were selling accesses to the tools:

“I am not involved in the creation or development of this bot. My role is to connect you to the source, and I must clarify that I hold no responsibility for how you utilize this tool. I am merely providing you with access, and any actions or consequences stemming from its use are solely your responsibility.”

FraudGPT was advertised in a known fraud Telegram channel, “t.me/TheCashFlowCartel”, where the admin of the channel, CanadianKingpin12, also offered carding and money laundering related services. The same day, CanadianKingpin12 published on a known carding forum about their new AI services, where they described an AI tool like ChatGPT specifically designed with no legal or ethical limitations, allowing any user to create malware, phishing campaigns and much more.

Shortly after FraudGPT was introduced in TheCashFlowCartel Telegram channel, on July 26, CanadianKingpin12 announced the sale of access to two new AI tools: DarkBARD and DarkBERT.

CanadinKinping12 publishing the AI tool

Figure 7: CanadinKinping12 publishing the AI tool.

Outpost24’s KrakenLabs analysts found that before that holiday period, CanadianKingpin12 was also known as CanadianSmoker12, as they themselves remarked in a remainder posted in a known carding market:

CanadianKinping12 confirming that they were CanadianSmoker12

Figure 8: CanadianKinping12 confirming that they were CanadianSmoker12.

Digging into this new moniker showed that CanadianKingpin12 had been selling illegal services since at least 2019:

 “Your #1 Fraud Vendor has returned from a well needed vacation! For those who are unaware ive been a top ranking vendor since 2019 dominating the most nutorious marketplaces across the globe (ToRReZ, World, Kingdom, Vice City, WHM, Alpha Bay) to name a few… with a track record of over 3,000+ confirmed sales and 1,000+ confirmed positive reviews (…).”

We’d like to note that even though no direct relation has been found between Laste (seller of WormGPT) and CanadianKingping12, we would like to mention that Outpost24’s KrakenLabs analysts found a Telegram profile named “t.me/Canadiankingpin” using the same profile picture of Laste in Exploit.In. The most likely explanation being a sloppy scammer trying to lure users into their trap.

Comparing images between the Laste profile and a Telegram user named “Canadian Kingpin"
Comparing images between the Laste profile and a Telegram user named “Canadian Kingpin

Figure 9: Comparing images between the Laste profile and a Telegram user named “Canadian Kingpin”.  

All that glitters is not gold

A tool quickly achieving popularity is not always good, as with a quick rise also comes an increased chance of something going wrong. It could either be that the infrastructure used becomes the target of a DDoS attack or even that researchers focus their investigations on the users behind its creation and end up being doxed and exposed.

The latter is exactly what happened on this occasion, as Outpost24’s KrakenLabs analysts observed that certain users and channels promoting these AI tools began to systematically remove any indicators linking them to their projects, effectively pretending nothing had occurred.

Since August 3rd, CanadianKingpin12 had systematically deleted all traces of them ever selling or announcing any AI tools, including FraudGPT, DarkBARD, DarkBERT and DarkGPT. The announcements about FraudGPT and other tools made in its Telegram channel, “t.me/TheCashFlowCartel”, can no longer be seen. Their posts in known Dark web forums have also been edited into nothingness, and the YouTube videos uploaded by the user showing the capabilities of the tools have disappeared. The reason for these actions is currently unknown. 

posts created by “canadiansmoker” AKA “CanadianKingpin12

Figure 10: Image showing all the posts created by “canadiansmoker” AKA “CanadianKingpin12” being reacted.

Moreover, a channel created to promote DarkBERT on July 25 (“t.me/DarkBERTOfficial”) has also been left without an owner since the admin user appeared as a deleted account.

Admin status of DarkBERTOfficial

Figure 11: Admin status of DarkBERTOfficial.

Last also announced on August 9th the end of the WormGPT project via their Telegram channel and the open forum threads, alluding to the level of visibility they got. All the videos, Pastebins, and other proofs that Last shared were removed. Despite the user’s alleged reasons for ending the project, it must be highlighted that the decision was announced after researcher Brian Krebs published an article covering WormGPT and revealing the real name and nationality of the individual behind Last [1]. Bearing this timeline of events, it is now in the hands of the reader to draw their own conclusions.

closing of the project in the WormGPT Telegram channel

Figure 12: Image showing the closing of the project in the WormGPT Telegram channel “t.me/wormgptdevelopment”.

Both Telegram channels controlled by Last and Darkstux were marked as closed the same day this announcement was made. Therefore, all these actions appear to confirm the hypothesis that there was a connection between both users.

WormGPT project closed.

Figure 13:  WormGPT project closed.

So, in an unexpected turn for the clients of these tools, despite forum posts suggesting that some of them had purchased yearly and lifetime licenses, none of the tools remained functional for more than a month and a half.

What next for dark AI tools?

Because some Large Language Models (LLM) are open source, anyone with enough knowledge can train them to create specifically tailored AI for fraudsters and cybercriminals, as was the case with GPT-J and WormGPT. Closed-access models are not safe either, as someone could gain access to them and start reselling it to other cyber-delinquents, as was the case with CanadianKingpin12

Moreover, as has been previously mentioned in this post, the hype surrounding AI tools has also attracted many scammers who have set up websites and Telegram channels to deceive people into purchasing nonexistent access to those tools. The rise in these scams is even more indicative of these emergent crime-enabling AI’s popularity and interest from the underground community.

As confirmed by Last on their farewell letter, anyone could reproduce what WormGPT did, so a likely new real business where scammers, developers, sellers and buyers should coexist could be glimpsed:

“At the end of the day, WormGPT is nothing more than an unrestricted ChatGPT. Anyone on the internet can employ a well-known jailbreak technique and achieve the same, if not better, results by using jailbroken versions of ChatGPT. In fact, being aware that we utilize GPT-J 6B as the language model, anyone can utilize the same uncensored model and achieve similar outcomes to those of WormGPT.”

We have seen the swift rise and fall of these AI tools with enormous potential in the wrong hands. These tools have demonstrated remarkable profitability, so they are likely not the last of their kind, as we already saw users trying to buy the WormGPT project moments after Last published the closure of the project.

User showing interest in buying WormGPT project after sudden closure

Figure 14:  User showing interest in buying WormGPT project after sudden closure

Hence, it’s reasonable to anticipate a rapid increase in both new dark AI tools and users, given the inherent profitability clearly recognized by the underground ecosystem. It did not take much for new tools to claim to be “the next WormGPT”, such as “Evil-GPT” and “Wolf-GPT”. These new tools, albeit much simpler than WormGPT was, underscore the widespread interest in capitalizing on this emerging and lucrative underground business opportunity.

About KrakenLabs

KrakenLabs is Outpost24’s Cyber Threat Intelligence team. Our team helps businesses stay ahead of malicious actors in the ever-evolving threat landscape, helping you keep your assets and brand reputation safe. With a comprehensive threat hunting infrastructure, our Threat Intelligence solution covers a broad range of threats on the market to help your business detect and deter external threats.

Learn more about Threat Compass

About the Author

Margarita Del Val
Margarita Del Val Threat Intelligence Analyst, Outpost24

Margarita is a seasoned Threat Intelligence Analyst with over five years of experience in the field. She is currently a part of the Strategic Research Team at KrakenLabs. With a background in Criminology, Margarita is deeply passionate about delving into the cybercrime ecosystem to comprehend the psychology behind threat actors and their motivations.