Master thesis : Novel LLM benchmark for test case generation

Job Description:

Join our Team

About this opportunity:

Are you ready for pushing limits and be part of creating the future?

Approximately 40% of the world's mobile traffic is carried through an (COMPANY NAME) network. (COMPANY NAME) is recognized technology leader in bringing it to market the 5th generation (5G) of Radio Access Networks (RAN) and Mobile Core Networks. The foundation for 5G use cases is built through an agile cloud environment that is distributed throughout the coverage area of the network. Software Defined Infrastructure is the automated equipment management layer that is key for enabling an agile private cloud, and our team is responsible for the system and software architecture that makes this happen.

What you will do:

Testing plays a key role in software development activities, enabling early detection of defects or undesirable effects therefore giving the development team opportunities to address them before the software is released for wide use. Recently, the potential for Large Language Models (LLMs) trained on code to help with different use cases related to software testing was identified as an important research area within software engineering.
While for code generation-related activities there are many benchmarks that enable to compare the results generated by different LLMs, there are very few such benchmarks that focus on testing-related tasks. This research proposes the development of a novel benchmark library for LLM outputs related to testing tasks that is well adapted to specific testing frameworks actively used in software development at (COMPANY NAME).

Objectives:

- Develop a novel benchmark for LLM output related to testing tasks in software development
- Evaluate the benchmark with at least one on-premises executed LLM
- Provide guidelines for adding new tasks or use cases to the benchmark for a specific programming language

The skills you bring:

You are a student at a Swedish university with a specialization preferably in computer science or software engineering. Background on data science is preferred, as well as excellent programming abilities in Java, good knowledge of Linux and Python relevant libraries (e.g., pandas, NumPy, scikit-learn, TensorFlow, PyTorch).

​What happens once you apply?
Click Here to find all you need to know about what our typical hiring process looks like.
We encourage you to consider applying to jobs where you might not meet all the criteria. We recognize that we all have transferrable skills, and we can support you with the skills that you need to develop.
Encouraging a diverse and inclusive organization is core to our values at (COMPANY NAME), that's why we champion it in everything we do. We truly believe that by collaborating with people with different experiences we drive innovation, which is essential for our future growth. We encourage people from all backgrounds to apply and realize their full potential as part of our (COMPANY NAME) team. (COMPANY NAME) is proud to be an Equal Opportunity Employer. learn more.
Primary country and city: Sweden (SE) || Stockholm
Job Stage: Job Stage 4
Primary Recruiter: Arvid Bergström