As new technologies like low-code/no-code development and generative AI (GenAI) revolutionize how we build and interact with software, they also bring about new security challenges—especially for the public sector. Protecting sensitive information and online accounts is more critical than ever, as cybercriminals look to exploit gaps in these emerging systems. Ensuring robust security and threat visibility is now essential for safeguarding against the risks associated with these advancements, especially as traditional safeguards become less effective in the face of evolving threats.
Low-code Development Exposes New Risk
One of the unintended consequences of our shift to a low-code/no-code development paradigm is the delegation of complex development tasks to Large Language Models (LLMs) and GenAI systems, often bypassing seasoned developers and architects. This opens new opportunities for cybercriminals. These systems excel at functional requirements—‘Build me a website that accepts customer checkout requests’—but they rarely infer non-functional needs, like security, unless explicitly instructed.
In traditional software development, security considerations are often implicit, stemming from the experience of developers and architects who’ve spent years learning from real-world failures. GenAI, however, lacks this depth of experience and focuses narrowly on the task at hand. The result? Incomplete or inadequate security measures in software developed through these systems. As organizations lean more heavily on GenAI, we risk creating an insecure software ecosystem ripe for exploitation by threat actors.
The Proliferation of Knowledge-Based Verification Attacks
We’re on the brink of a surge in automated attacks exploiting vulnerabilities in Knowledge-Based Verification (KBV) systems. Large-scale data breaches, like the one that exposed millions of Social Security numbers last year, are eroding the effectiveness of this approach at confirming identity when creating an account or supporting a password reset. These processes often rely on KBV—such as answering questions about your mother’s maiden name or the street you grew up on—but this information is increasingly accessible to malicious actors.
As these personal details become more widely available through data breaches and online marketplaces, attackers can easily bypass KBV systems. Worse yet, threat actors can now leverage LLMs to develop sophisticated tools to mine personal data at scale and orchestrate automated attacks against these KBV systems. Organizations face an urgent challenge: how to protect accounts in a world where traditional KBV methods are no longer secure or reliable while still offering users a legitimate path to create an account or regain access when needed.
LLM Safeguards Can Be Overridden or Bypassed by Running Models Locally
With the proliferation of local LLM instances and tools like Ollama, we’ll see safeguards embedded in commercial LLMs eroded or bypassed entirely. Running models locally can allow threat actors to fine-tune them, removing restrictions on malicious activity and enabling custom models optimized for cybercrime. This creates a new frontier for scaled attacks that are faster, more targeted, and harder to detect until it’s too late.
Imagine a threat actor fine-tuning a model to craft phishing campaigns, identify vulnerabilities in software, or automate account takeovers. The ability to localize and modify these models fundamentally shifts the balance, empowering attackers with tools tailored to their malicious intent. The guardrails built into commercial LLMs are no match for this growing trend, amplifying the need for robust detection and defense strategies at every level.
As the public sector continues to adopt innovative technologies, staying ahead of emerging cyber threats is crucial. The increasing sophistication of attacks, such as those targeting KBV systems and leveraging GenAI, highlights the need for stronger protections. By prioritizing comprehensive security measures and threat detection, organizations can mitigate the risks of these evolving vulnerabilities and safeguard their sensitive data and online accounts against malicious actors. It is essential to build and maintain resilient security strategies to ensure the integrity of digital infrastructures in this rapidly changing environment.
To learn more about how HUMAN Security helps the public sector protect citizen accounts, sensitive information, and critical infrastructure, click here.
Carahsoft Technology Corp. is The Trusted Government IT Solutions Provider, supporting Public Sector organizations across Federal, State and Local Government agencies and Education and Healthcare markets. As the Master Government Aggregator for our vendor partners, including HUMAN Security, we deliver solutions for Geospatial, Cybersecurity, MultiCloud, DevSecOps, Artificial Intelligence, Customer Experience and Engagement, Open Source and more. Working with resellers, systems integrators and consultants, our sales and marketing teams provide industry leading IT products, services and training through hundreds of contract vehicles. Explore the Carahsoft Blog to learn more about the latest trends in Government technology markets and solutions, as well as Carahsoft’s ecosystem of partner thought-leaders.
