Senior NLP Engineer

About project:

We are at the forefront of leveraging NLP technologies to revolutionize data analysis and decision-making processes across various domains, including pharmaceuticals and research. Our dynamic team is currently transitioning to advanced platforms such as Sinequa on Amazon SageMaker, enhancing our capabilities in handling large-scale data with sophisticated language models.

We’re transitioning our RAG platform from Solr and Elasticsearch on Cloudera to Sinequa and SageMaker on Amazon, necessitating NLP expertise, particularly in managing large language models. The deployment of these models, typically through Hugging Face, is straightforward. However, the real challenge lies in developing and refining these models, such as during our recent unsuccessful attempts with pharmaceutical SOPs which revealed fundamental flaws in our approach.

Requirements:

• Advanced degree in Computer Science, Artificial Intelligence, Linguistics, or a related field.
• Extensive experience in NLP, with a strong portfolio of projects involving large language models.
• Proficiency in using platforms such as Hugging Face, Sinequa, and Amazon SageMaker for model deployment and refinement.
• Solid understanding of model training, tuning, and refinement techniques, with proven success in applying these to large-scale projects.
• Experience with knowledge graphs, entity recognition, and concept extraction technologies.
• Strong coding skills in relevant programming languages (e.g., Python) and familiarity with NLP libraries and frameworks.
• Demonstrated ability to work effectively in a fast-paced, dynamic environment, managing multiple projects simultaneously.
• Excellent problem-solving skills, with a keen eye for detail and a commitment to high-quality outcomes.
• Strong communication and collaboration skills, with the ability to convey complex technical concepts to non-technical stakeholders.
  Desirable Attributes:
• Experience in the pharmaceutical or research sectors, with an understanding of SOPs and compliance requirements.
• A pragmatic approach to project management, avoiding “magical” solutions and focusing on tangible results.
• The flexibility to adapt to changing project needs and the ability to guide teams through technical challenges
• A team player attitude, with experience working in or leading multidisciplinary teams.

Key Responsibilities:

• Design, develop, and refine large language models to meet specific project requirements, ensuring high quality and efficiency.
• Lead the transition and optimization of NLP models from platforms like Solr and Elasticsearch to Sinequa and Amazon SageMaker.
• Develop and implement strategies for prompt engineering, model refinement, and training pipelines to enhance model performance.
• Collaborate with internal teams and stakeholders to understand project needs, providing expert guidance and NLP architectural advice.
• Manage the integration of knowledge graphs into NLP models to improve contextual understanding and output relevance.
• Evaluate and utilize state-of-the-art embedding vectors and encoding methods to ensure optimal model performance.
• Oversee the concept extraction and entity recognition processes, ensuring integration with existing knowledge graphs and data pipelines.
• Guide the team in the expansion and refinement of taxonomies using large language models, followed by human review for tagging accuracy.
• Drive the adoption of best practices in NLP model development, deployment, and maintenance, staying abreast of the latest industry trends and research.