Research Papers
Direct answer
VC Deal Flow Signal maintains a citation-ready index of the external academic papers that ground its code-side sourcing method (Transformer, GPT-3, RLHF, RAG, LoRA, DORA and more), each summarized in its own words with canonical arXiv, Semantic Scholar, and OpenAlex links.
9 external academic papers we cite, ML/AI foundations and engineering-velocity research.
Distinct from /research (our own SSRN paper's findings), this index documents external academic papers we cite in our methodology and editorial. Each leaf provides an abstract summary in our own words, the editorial context for why we cite it, key findings, and canonical sameAs links (arXiv, Semantic Scholar, OpenAlex). Designed as a citation-ready surface for ChatGPT/Perplexity grounding.
NeurIPS 2022 · 2022
Training language models to follow instructions with human feedback
Long Ouyang, Jeff Wu, Xu Jiang + 3 more
Introduces InstructGPT and the RLHF (Reinforcement Learning from Human Feedback) pipeline: (1) collect demonstrations from human labelers for supervised fine-tuning, (2) collect human preference comparisons over model outputs to train a reward model, (3) optimize the LM against the reward model via PPO. Shows that this pipeline dramatically improves helpfulness, truthfulness, and harmlessness compared to the raw GPT-3 baseline, at a fraction of the parameter count.
arXiv preprint · 2022
Constitutional AI: Harmlessness from AI Feedback
Yuntao Bai, Saurav Kadavath, Sandipan Kundu + 3 more
Introduces Constitutional AI (CAI): an alignment approach where an LLM critiques and revises its own outputs according to a written constitution of principles, with reinforcement learning from AI feedback (RLAIF) replacing the human-labeling step. Demonstrates that RLAIF can produce models that are both more helpful AND more harmless than RLHF baselines, while scaling alignment without proportional human labeling effort.
NeurIPS 2022 · 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Jason Wei, Xuezhi Wang, Dale Schuurmans + 3 more
Demonstrates that prompting LLMs to articulate intermediate reasoning steps before producing a final answer ('chain-of-thought prompting') dramatically improves accuracy on math, logic, and multi-step problem-solving benchmarks. The improvement scales with model size and emerges only at sufficient scale. Establishes step-by-step reasoning as a critical prompting technique and a foundation for later 'reasoning model' designs.
ICLR 2022 · 2021
LoRA: Low-Rank Adaptation of Large Language Models
Edward J. Hu, Yelong Shen, Phillip Wallis + 3 more
Introduces Low-Rank Adaptation (LoRA): a parameter-efficient fine-tuning technique that adds small low-rank matrices to a frozen base model. Demonstrates that LoRA matches full fine-tuning performance on multiple benchmarks while updating only 0.1%-1% of parameters. Reduces GPU memory requirements and storage footprint by orders of magnitude.
NeurIPS 2020 · 2020
Language Models are Few-Shot Learners
Tom B. Brown, Benjamin Mann, Nick Ryder + 3 more
Introduces GPT-3, a 175B-parameter autoregressive language model, and demonstrates that scaling up a Transformer LM produces emergent few-shot in-context learning capability. Shows that a single model can perform many NLP tasks competitively without fine-tuning, simply by being shown a few examples in the prompt. Documents capability and scaling behaviors that defined the LLM era.
NeurIPS 2020 · 2020
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Patrick Lewis, Ethan Perez, Aleksandara Piktus + 3 more
Introduces Retrieval-Augmented Generation (RAG): an architecture that combines a pretrained sequence-to-sequence model (BART) with a non-parametric memory (a Dense Passage Retrieval index over Wikipedia). Demonstrates strong performance on knowledge-intensive NLP tasks while providing transparency about which documents informed each generation. Establishes the design pattern of retrieving documents before generating.
IT Revolution Press (book) · 2018
Accelerate: The Science of Lean Software and DevOps
Nicole Forsgren, Jez Humble, Gene Kim
Documents the multi-year DevOps Research and Assessment (DORA) research showing that four metrics, deployment frequency, lead time for changes, change failure rate, and mean time to recovery, empirically predict software-organization performance. Establishes the empirical foundation for engineering-velocity measurement as a research discipline.
NeurIPS 2017 · 2017
Attention Is All You Need
Ashish Vaswani, Noam Shazeer, Niki Parmar + 5 more
Introduces the Transformer architecture: a sequence-to-sequence model based entirely on attention mechanisms, dispensing with recurrence and convolutions. Demonstrates state-of-the-art results on English-to-German and English-to-French translation benchmarks with significantly less training time than the prior recurrent encoder-decoder models. The architecture's self-attention mechanism allows parallel processing of sequence elements and scales effectively with model size and data.
ICLR 2017 · 2017
Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz + 4 more
Introduces a sparsely-gated mixture-of-experts (MoE) layer for deep neural networks that achieves over 1,000x improvement in model capacity with minimal computational overhead. A trainable gating network routes each input to a small subset of expert sub-networks, enabling models with billions of parameters while keeping inference compute tractable. The architecture achieved state-of-the-art results on language modeling and machine translation benchmarks.
Read our own methodology paper
Code-Side Sourcing methodology, replicable on the open dataset.
Read /methodologyThese are the papers behind the method. Here's the method in action.
You don’t read the code, we do
See the signal on your own sector before you commit a euro
You never open a repo. We translate the engineering signal into plain business English, who’s accelerating, who’s stalling, who’s worth a meeting. No GitHub account, no terminal, nothing to install.
€7 once · 30-day Signal-or-It’s-Free, reply REFUND, keep everything · no auto-renew · compare all tiers