Hiring.Camp

Cybersecurity Engineer - AI Benchmarking

Gramian Consulting Group

·

Today

Location
Brazil · Bangladesh · Colombia · Egypt · Ghana · India
Workplace
Remote
Type
Contract
Department
Talent Solutions
Experience
5+ years
Source
Workable

Description

About Gramian

Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.

About the Role

We are seeking an experienced Senior Software Security Engineer to develop high-quality training and evaluation tasks for advanced AI coding models. You will transform real-world software vulnerabilities, including CVEs, security advisories, and open-source bug reports, into realistic Harbor-format audit-and-patch tasks across multiple programming languages.

This role combines software security research, vulnerability reproduction, benchmark development, automated testing, and technical review. You will create tasks that assess whether AI systems can identify vulnerabilities, understand their underlying causes, reproduce security issues, and implement reliable fixes rather than simply recognizing familiar patterns or copying existing patches.

The ideal candidate has strong professional software engineering experience, deep practical knowledge of application security, and the ability to independently investigate vulnerabilities and build reproducible testing environments.

Engagement Type: Contractor assignment
Duration: 3 months
Start Date: Expected to start next week
Commitment: 40 hours per week
Work Arrangement: Fully remote
Project Focus: AI coding model training and evaluation, software vulnerability research, and secure patch development.

Key Responsibilities

  • Vulnerability Research: Source and investigate real-world vulnerabilities from CVEs, security advisories, bug reports, and open-source repositories.
  • Security Classification: Identify and classify vulnerabilities using the most specific applicable CWE and map them to relevant OWASP Top 10 categories where appropriate.
  • Vulnerability Reproduction: Reproduce security issues and develop reliable proof-of-vulnerability tests that demonstrate the underlying weakness.
  • Benchmark Development: Build Harbor-format audit-and-patch tasks across multiple programming languages and software ecosystems.
  • Task Authoring: Write realistic task instructions that describe the security issue without revealing vulnerable code, exploit details, or the expected remediation.
  • Environment Setup: Create reproducible Docker-based environments with pinned dependencies, clear setup instructions, and reliable execution procedures.
  • Verifier Development: Design and harden automated verifiers to prevent gaming, test manipulation, hardcoded solutions, and overfitting.
  • Patch Evaluation: Review and score AI-generated patches and technical reasoning using structured security and engineering rubrics.
  • Quality Assurance: Identify tasks that can be solved through superficial pattern matching or copying known upstream patches without understanding the vulnerability.
  • Technical Documentation: Document vulnerability context, reproduction steps, expected behavior, patch requirements, test methodology, and review decisions.

Skills

PythonJavaScriptTypeScriptJavaRustRubyPHPKotlinDockerLinuxGitTechnical Writing

Similar Jobs

30

Cybersecurity Engineer

Wabtec·Suzhou, Jiangsu·Onsite

1d ago

Cybersecurity Engineer

Select Water Solutions·Houston, Texas

1d ago

Cybersecurity Engineer

Digital Check Corp·Northbrook, Illinois

1d ago

Cybersecurity Engineer

CACI Ltd·US·Onsite

1d ago

Cybersecurity Engineer

Mayvin, Inc·Fort Belvoir, Virginia·Hybrid

Cybersecurity Engineer

Rule4·Boulder, Colorado·Hybrid

5d ago

Cybersecurity Engineer

Tranetechnologies·US·Onsite

1w ago

Cybersecurity Engineer

Bertelsmann-Jobs·Romania·Hybrid

1w ago

Cybersecurity Engineer

VivSoft Technologies·Remote

1w ago

Cybersecurity Engineer

Visa·Austin, Texas·Hybrid

1w ago

Cybersecurity Engineer

Planet Technologies·Washington, District of Columbia·Onsite

1w ago

Cybersecurity Engineer

Talent Acquisition Team·Indianapolis, Indiana·Remote

1w ago

Cybersecurity Engineer

Professional Kyndryl·KSG51588 Singapore·Hybrid

1w ago

Cybersecurity Engineer

Georgia·US

1w ago

Cybersecurity Engineer

Gdit·US·Remote

1w ago

Cybersecurity Engineer 

Turium·Hyderabad, Telangana·Onsite

1w ago

Cybersecurity Engineer

Redwood Credit Union·Napa, California·Onsite

2w ago

Cybersecurity Engineer

VIDIZMO LLC·Karachi, Sindh

2w ago

Cybersecurity Engineer

TrekFed·Remote

Cybersecurity Engineer

Piedmont Healthcare Care·Statesville, North Carolina

2w ago

Cybersecurity Engineer

Clarios·Mexico·Hybrid

2w ago

Cybersecurity Engineer

SGS·Hanoi, Vietnam·Onsite

2w ago

Cybersecurity Engineer

Cmu·Pittsburgh, Pennsylvania

2w ago

Cybersecurity Engineer

"Omitron, Inc."·Colorado Springs, Colorado

2w ago

Cybersecurity Engineer

Sprawdź nasze oferty pracy

Cybersecurity Engineer

Nrel·New York City, New York·Remote

3w ago

Cybersecurity Engineer

i3 Broadband·East Peoria, Illinois +1

3w ago

Cybersecurity Engineer

Dscout·Remote·Remote

3w ago

Cybersecurity Engineer

Default (MSD)·Rockville, Maryland

3w ago

Cybersecurity Engineer

Cambridge Consultants·London, England·Hybrid

3w ago