About the Team The Safety Systems team is responsible for various safety work to ensure our best models can be safely deployed to the real world to benefit the society and is at the forefront of OpenAI's mission to build and deploy safe AGI, driving our commitment to AI safety and fostering a culture of trust and transparency. The Model Safety Research team aims to fundamentally advance our capabilities for precisely implementing robust, safe behavior in AI models, and to leverage these advances to make OpenAI's deployed models safe and beneficial. This requires a breadth of new ML research to address the growing set of safety challenges as AI becomes more powerful and used in more settings. Key focus areas include how to enforce nuanced safety policies without trading off helpfulness and capabilities, how to make the model robust to adversaries, how to address privacy and security risks, and how to make the model trustworthy in safety-critical domains. We seek to learn from deployment and distribute the benefits of AI, while ensuring that this powerful tool is used responsibly and safely. About the Role OpenAI is seeking a senior researcher with passion for AI safety and experience in safety research. Your role will set directions for research to enable and empower safe AGI and work on research projects to make our AI systems safer, more aligned and more robust to adversarial or malicious use cases. You will play a critical role in shaping how a safe AI system should look like in the future at OpenAI, making a significant impact on our mission to build and deploy safe AGI. In this role, you will: <ul><li>Conduct state-of-the-art research on AI safety topics such as RLHF, adversarial training, robustness, and more.</li><li>Implement new methods in OpenAI's core model training and launch safety improvements in OpenAI's products.</li><li>Set the research directions and strategies to make our AI systems safer, more aligned and more robust.</li><li>Coordinate and collaborate with cross-functional teams, including T&S, legal, policy and other research teams, to ensure that our products meet the highest safety standards.</li><li>Actively evaluate and understand the safety of our models and systems, identifying areas of risk and proposing mitigation strategies.</li></ul>You might thrive in this role if you: <ul><li>Are excited about OpenAI's mission of building safe, universally beneficial AGI and are aligned with OpenAI's charter</li><li>Demonstrate a passion for AI safety and making cutting-edge AI models safer for real-world use.</li><li>Bring 4+ years of experience in the field of AI safety, especially in areas like RLHF, adversarial training, robustness, fairness & biases.</li><li>Hold a Ph.D. or other degree in computer science, machine learning, or a related field.</li><li>Possess experience in safety work for AI model deployment</li><li>Have an in-depth understanding of deep learning research and/or strong engineering skills.</li><li>Are a team player who enjoys collaborative work environments.</li></ul> About OpenAI OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity. We are an equal opportunity employer and do not discriminate on the basis of race, religion, national origin, gender, sexual orientation, age, veteran status, disability or any other legally protected status. For US Based Candidates: Pursuant to the San Francisco Fair Chance Ordinance, we will consider qualified applicants with arrest and conviction records. We are committed to providing reasonable accommodations to applicants with disabilities, and requests can be made via this link. OpenAI Global Applicant Privacy Policy At OpenAI, we believe artificial intelligence has the potential to help people solve immense global challenges, and we want the upside of AI to be widely shared. Join us in shaping the future of technology.

OpenAIs Superalignment Team is working on technical approaches to ensure that superintelligencean AI system vastly smarter than humansfollows human intent.Through scientific experimentation, we explore the scalability of alignment techniques and identify potential breaking points. Our approach to alignment research includes a range of different projects; some of these will help us improve the alignment of our models and others will allow us to validate how aligned our models actually are:Generalization: Can we understand and control how our models generalize from easy tasks that humans can supervise to hard tasks that humans cannot?Robustness: How can we train our models to be aligned in worst-case situations?Adversarial testing: If we deliberately train deceptively aligned models as testbeds, can our oversight techniques, interpretability tools, and evaluations detect this misalignment?We want to figure out how to spend vast amounts of compute to solve this problem, in particular by automating alignment research itself.About the RoleAs a Research Scientist here, you will develop innovative machine learning techniques and advance the research agenda of the Superalignment team, while also collaborating with peers across the organization. We are looking for people who want to discover simple, generalizable ideas that work well even at large scale, and form part of a broader research vision that unifies the entire company.We are seeking Research Scientists to help design and implement experiments for alignment research. Responsibilities may include:Designing experiments to measure the effectiveness of scalable oversight techniques such as AI-assisted feedback and DebateStudying generalization to see when AI systems trained on easy problems can solve hard problemsManaging large datasets from interpretability experiments and creating visualizations to explore interpretability dataDeveloping experiments to test how well chain of thought reasoning reflects model cognitionInvestigating situations when training against a reward signal causes model outputs to deteriorateExploring methods to understand and predict model behaviors, such as finding inputs causing anomalous circuits or catastrophic outputsDesigning novel approaches for using LLMs in alignment researchHave a track record of coming up with new ideas or improving upon existing ideas in machine learning, demonstrated by accomplishments such as first author publications or projectsPossess the ability to own and pursue a research agenda, including choosing impactful research problems and autonomously carrying out long-running projectsPossess a strong curiosity about aligning and understanding ML models, and are motivated to use your career to address this challengeEnjoy fast-paced, collaborative, and cutting-edge research environmentsHave experience implementing ML algorithms (e.g., PyTorch)Want to ensure that powerful AI systems stay under human controlAbout OpenAIOpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity.We are an equal opportunity employer and do not discriminate on the basis of race, religion, national origin, gender, sexual orientation, age, veteran status, disability or any other legally protected status.For US Based Candidates: Pursuant to the San Francisco Fair Chance Ordinance, we will consider qualified applicants with arrest and conviction records.We are committed to providing reasonable accommodations to applicants with disabilities, and requests can be made via thislink .At OpenAI, we believe artificial intelligence has the potential to help people solve immense global challenges, and we want the upside of AI to be widely shared. Join us in shaping the future of technology.#J-18808-Ljbffrby Jobble

About the Team The Post-training Core Model team evaluates and improves the capability of the models that power ChatGPT and the OpenAI API. About the Role As a Member of Technical Staff on our team, you will research and develop improvements to our models through human data collection, synthetic data studies, supervised fine-tuning, reward modeling, and evaluations. We're looking for people who have past scientific experience, are proficient software engineers, and have experience in reinforcement learning or reinforcement learning with human feedback (RLHF) research. This role is based in San Francisco, CA. We use a hybrid work model of 3 days in the office per week and offer relocation assistance to new employees. In this role, you will: <ul><li>Design, validate, and improve evaluations</li><li>Conduct human data campaigns closely informed by development of new RLHF algorithms</li><li>Conduct and present on machine learning experiments</li><li>Prototype new ideas and algorithms</li><li>Launch your innovations into ChatGPT and the API</li></ul> You might thrive in this role if you: <ul><li>Are a careful scientist and a strong software engineer who wants research with real-world impact</li><li>Aren't scared of the messiness of handling real-world data</li><li>Can navigate complicated code bases</li><li>Are an effective collaborator and can communicate technical work clearly</li><li>Can iterate quickly on lots of ideas and algorithms</li><li>Love being on the cutting edge of RL and language model research</li></ul> About OpenAI OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity. We are an equal opportunity employer and do not discriminate on the basis of race, religion, national origin, gender, sexual orientation, age, veteran status, disability or any other legally protected status. For US Based Candidates: Pursuant to the San Francisco Fair Chance Ordinance, we will consider qualified applicants with arrest and conviction records. We are committed to providing reasonable accommodations to applicants with disabilities, and requests can be made via this link. OpenAI Global Applicant Privacy Policy At OpenAI, we believe artificial intelligence has the potential to help people solve immense global challenges, and we want the upside of AI to be widely shared. Join us in shaping the future of technology.

Research Scientist Openai Jobs

Research Scientist Model Safety
April 30, 2024

Research Scientist Superalignment Openai San Francisco Bay Area Usa
April 21, 2024

Research Scientist Superalignment Openai San Francisco Bay Area Usa
April 18, 2024

Research Scientist Core Model Posttraining
April 4, 2024

Research Scientist
April 3, 2024

Research Scientist Model Safety
April 3, 2024

Research Scientist Superalignment
April 3, 2024

Quick filters

Top companies

Research Scientist Openai Jobs

Research Scientist Model SafetyApril 30, 2024

Research Scientist Superalignment Openai San Francisco Bay Area UsaApril 21, 2024

Research Scientist Superalignment Openai San Francisco Bay Area UsaApril 18, 2024

Research Scientist Core Model PosttrainingApril 4, 2024

Research ScientistApril 3, 2024

Research Scientist Model SafetyApril 3, 2024

Research Scientist SuperalignmentApril 3, 2024