The agent engineering platform.
Python agentsaiai-agentsanthropicchatgptdeepagentsenterpriseframeworkgeminigenerative-ailangchainlanggraphllmmultiagentopen-sourceopenaipydanticpythonragtypescript
docs: `arxiv` page update (#25450)
Leonid Ganeline8eb63a609e2e
diff --git a/docs/docs/additional_resources/arxiv_references.mdx b/docs/docs/additional_resources/arxiv_references.mdxindex 8b1b84c324..b9848168e5 100644--- a/docs/docs/additional_resources/arxiv_references.mdx+++ b/docs/docs/additional_resources/arxiv_references.mdx@@ -4,8 +4,11 @@ LangChain implements the latest research in the field of Natural Language ProcesThis page contains `arXiv` papers referenced in the LangChain Documentation, API Reference,Templates, and Cookbooks.-From the opposite direction, scientists use LangChain in research and reference LangChain in the research papers.-Here you find [such papers](https://arxiv.org/search/?query=langchain&searchtype=all&source=header).+From the opposite direction, scientists use `LangChain` in research and reference it in the research papers.+Here you find papers that reference:+- [LangChain](https://arxiv.org/search/?query=langchain&searchtype=all&source=header)+- [LangGraph](https://arxiv.org/search/?query=langgraph&searchtype=all&source=header)+- [LangSmith](https://arxiv.org/search/?query=langsmith&searchtype=all&source=header)## Summary@@ -23,32 +26,30 @@ Here you find [such papers](https://arxiv.org/search/?query=langchain&searchtype| `2305.14283v3` [Query Rewriting for Retrieval-Augmented Large Language Models](http://arxiv.org/abs/2305.14283v3) | Xinbei Ma, Yeyun Gong, Pengcheng He, et al. | 2023-05-23 | `Template:` [rewrite-retrieve-read](https://python.langchain.com/docs/templates/rewrite-retrieve-read), `Cookbook:` [rewrite](https://github.com/langchain-ai/langchain/blob/master/cookbook/rewrite.ipynb)| `2305.08291v1` [Large Language Model Guided Tree-of-Thought](http://arxiv.org/abs/2305.08291v1) | Jieyi Long | 2023-05-15 | `API:` [langchain_experimental.tot](https://api.python.langchain.com/en/latest/experimental_api_reference.html#module-langchain_experimental.tot), `Cookbook:` [tree_of_thought](https://github.com/langchain-ai/langchain/blob/master/cookbook/tree_of_thought.ipynb)| `2305.04091v3` [Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models](http://arxiv.org/abs/2305.04091v3) | Lei Wang, Wanyu Xu, Yihuai Lan, et al. | 2023-05-06 | `Cookbook:` [plan_and_execute_agent](https://github.com/langchain-ai/langchain/blob/master/cookbook/plan_and_execute_agent.ipynb)+| `2305.02156v1` [Zero-Shot Listwise Document Reranking with a Large Language Model](http://arxiv.org/abs/2305.02156v1) | Xueguang Ma, Xinyu Zhang, Ronak Pradeep, et al. | 2023-05-03 | `API:` [langchain...LLMListwiseRerank](https://api.python.langchain.com/en/latest/retrievers/langchain.retrievers.document_compressors.listwise_rerank.LLMListwiseRerank.html#langchain.retrievers.document_compressors.listwise_rerank.LLMListwiseRerank)| `2304.08485v2` [Visual Instruction Tuning](http://arxiv.org/abs/2304.08485v2) | Haotian Liu, Chunyuan Li, Qingyang Wu, et al. | 2023-04-17 | `Cookbook:` [Semi_structured_and_multi_modal_RAG](https://github.com/langchain-ai/langchain/blob/master/cookbook/Semi_structured_and_multi_modal_RAG.ipynb), [Semi_structured_multi_modal_RAG_LLaMA2](https://github.com/langchain-ai/langchain/blob/master/cookbook/Semi_structured_multi_modal_RAG_LLaMA2.ipynb)| `2304.03442v2` [Generative Agents: Interactive Simulacra of Human Behavior](http://arxiv.org/abs/2304.03442v2) | Joon Sung Park, Joseph C. O'Brien, Carrie J. Cai, et al. | 2023-04-07 | `Cookbook:` [multiagent_bidding](https://github.com/langchain-ai/langchain/blob/master/cookbook/multiagent_bidding.ipynb), [generative_agents_interactive_simulacra_of_human_behavior](https://github.com/langchain-ai/langchain/blob/master/cookbook/generative_agents_interactive_simulacra_of_human_behavior.ipynb)| `2303.17760v2` [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](http://arxiv.org/abs/2303.17760v2) | Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, et al. | 2023-03-31 | `Cookbook:` [camel_role_playing](https://github.com/langchain-ai/langchain/blob/master/cookbook/camel_role_playing.ipynb)| `2303.17580v4` [HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face](http://arxiv.org/abs/2303.17580v4) | Yongliang Shen, Kaitao Song, Xu Tan, et al. | 2023-03-30 | `API:` [langchain_experimental.autonomous_agents](https://api.python.langchain.com/en/latest/experimental_api_reference.html#module-langchain_experimental.autonomous_agents), `Cookbook:` [hugginggpt](https://github.com/langchain-ai/langchain/blob/master/cookbook/hugginggpt.ipynb)-| `2303.08774v6` [GPT-4 Technical Report](http://arxiv.org/abs/2303.08774v6) | OpenAI, Josh Achiam, Steven Adler, et al. | 2023-03-15 | `Docs:` [docs/integrations/vectorstores/mongodb_atlas](https://python.langchain.com/docs/integrations/vectorstores/mongodb_atlas)-| `2301.10226v4` [A Watermark for Large Language Models](http://arxiv.org/abs/2301.10226v4) | John Kirchenbauer, Jonas Geiping, Yuxin Wen, et al. | 2023-01-24 | `API:` [langchain_community...HuggingFaceEndpoint](https://api.python.langchain.com/en/latest/llms/langchain_community.llms.huggingface_endpoint.HuggingFaceEndpoint.html#langchain_community.llms.huggingface_endpoint.HuggingFaceEndpoint), [langchain_huggingface...HuggingFaceEndpoint](https://api.python.langchain.com/en/latest/llms/langchain_huggingface.llms.huggingface_endpoint.HuggingFaceEndpoint.html#langchain_huggingface.llms.huggingface_endpoint.HuggingFaceEndpoint), [langchain_community...OCIModelDeploymentTGI](https://api.python.langchain.com/en/latest/llms/langchain_community.llms.oci_data_science_model_deployment_endpoint.OCIModelDeploymentTGI.html#langchain_community.llms.oci_data_science_model_deployment_endpoint.OCIModelDeploymentTGI), [langchain_community...HuggingFaceTextGenInference](https://api.python.langchain.com/en/latest/llms/langchain_community.llms.huggingface_text_gen_inference.HuggingFaceTextGenInference.html#langchain_community.llms.huggingface_text_gen_inference.HuggingFaceTextGenInference)+| `2301.10226v4` [A Watermark for Large Language Models](http://arxiv.org/abs/2301.10226v4) | John Kirchenbauer, Jonas Geiping, Yuxin Wen, et al. | 2023-01-24 | `API:` [langchain_community...OCIModelDeploymentTGI](https://api.python.langchain.com/en/latest/llms/langchain_community.llms.oci_data_science_model_deployment_endpoint.OCIModelDeploymentTGI.html#langchain_community.llms.oci_data_science_model_deployment_endpoint.OCIModelDeploymentTGI), [langchain_huggingface...HuggingFaceEndpoint](https://api.python.langchain.com/en/latest/llms/langchain_huggingface.llms.huggingface_endpoint.HuggingFaceEndpoint.html#langchain_huggingface.llms.huggingface_endpoint.HuggingFaceEndpoint), [langchain_community...HuggingFaceEndpoint](https://api.python.langchain.com/en/latest/llms/langchain_community.llms.huggingface_endpoint.HuggingFaceEndpoint.html#langchain_community.llms.huggingface_endpoint.HuggingFaceEndpoint), [langchain_community...HuggingFaceTextGenInference](https://api.python.langchain.com/en/latest/llms/langchain_community.llms.huggingface_text_gen_inference.HuggingFaceTextGenInference.html#langchain_community.llms.huggingface_text_gen_inference.HuggingFaceTextGenInference)| `2212.10496v1` [Precise Zero-Shot Dense Retrieval without Relevance Labels](http://arxiv.org/abs/2212.10496v1) | Luyu Gao, Xueguang Ma, Jimmy Lin, et al. | 2022-12-20 | `API:` [langchain...HypotheticalDocumentEmbedder](https://api.python.langchain.com/en/latest/chains/langchain.chains.hyde.base.HypotheticalDocumentEmbedder.html#langchain.chains.hyde.base.HypotheticalDocumentEmbedder), `Template:` [hyde](https://python.langchain.com/docs/templates/hyde), `Cookbook:` [hypothetical_document_embeddings](https://github.com/langchain-ai/langchain/blob/master/cookbook/hypothetical_document_embeddings.ipynb)| `2212.07425v3` [Robust and Explainable Identification of Logical Fallacies in Natural Language Arguments](http://arxiv.org/abs/2212.07425v3) | Zhivar Sourati, Vishnu Priya Prasanna Venkatesh, Darshan Deshpande, et al. | 2022-12-12 | `API:` [langchain_experimental.fallacy_removal](https://api.python.langchain.com/en/latest/experimental_api_reference.html#module-langchain_experimental.fallacy_removal)| `2211.13892v2` [Complementary Explanations for Effective In-Context Learning](http://arxiv.org/abs/2211.13892v2) | Xi Ye, Srinivasan Iyer, Asli Celikyilmaz, et al. | 2022-11-25 | `API:` [langchain_core...MaxMarginalRelevanceExampleSelector](https://api.python.langchain.com/en/latest/example_selectors/langchain_core.example_selectors.semantic_similarity.MaxMarginalRelevanceExampleSelector.html#langchain_core.example_selectors.semantic_similarity.MaxMarginalRelevanceExampleSelector)-| `2211.10435v2` [PAL: Program-aided Language Models](http://arxiv.org/abs/2211.10435v2) | Luyu Gao, Aman Madaan, Shuyan Zhou, et al. | 2022-11-18 | `API:` [langchain_experimental...PALChain](https://api.python.langchain.com/en/latest/pal_chain/langchain_experimental.pal_chain.base.PALChain.html#langchain_experimental.pal_chain.base.PALChain), [langchain_experimental.pal_chain](https://api.python.langchain.com/en/latest/experimental_api_reference.html#module-langchain_experimental.pal_chain), `Cookbook:` [program_aided_language_model](https://github.com/langchain-ai/langchain/blob/master/cookbook/program_aided_language_model.ipynb)-| `2210.03629v3` [ReAct: Synergizing Reasoning and Acting in Language Models](http://arxiv.org/abs/2210.03629v3) | Shunyu Yao, Jeffrey Zhao, Dian Yu, et al. | 2022-10-06 | `Docs:` [docs/integrations/providers/cohere](https://python.langchain.com/docs/integrations/providers/cohere), [docs/integrations/chat/huggingface](https://python.langchain.com/docs/integrations/chat/huggingface), [docs/integrations/tools/ionic_shopping](https://python.langchain.com/docs/integrations/tools/ionic_shopping), `API:` [langchain...create_react_agent](https://api.python.langchain.com/en/latest/agents/langchain.agents.react.agent.create_react_agent.html#langchain.agents.react.agent.create_react_agent), [langchain...TrajectoryEvalChain](https://api.python.langchain.com/en/latest/evaluation/langchain.evaluation.agents.trajectory_eval_chain.TrajectoryEvalChain.html#langchain.evaluation.agents.trajectory_eval_chain.TrajectoryEvalChain)+| `2211.10435v2` [PAL: Program-aided Language Models](http://arxiv.org/abs/2211.10435v2) | Luyu Gao, Aman Madaan, Shuyan Zhou, et al. | 2022-11-18 | `API:` [langchain_experimental.pal_chain](https://api.python.langchain.com/en/latest/experimental_api_reference.html#module-langchain_experimental.pal_chain), [langchain_experimental...PALChain](https://api.python.langchain.com/en/latest/pal_chain/langchain_experimental.pal_chain.base.PALChain.html#langchain_experimental.pal_chain.base.PALChain), `Cookbook:` [program_aided_language_model](https://github.com/langchain-ai/langchain/blob/master/cookbook/program_aided_language_model.ipynb)+| `2210.03629v3` [ReAct: Synergizing Reasoning and Acting in Language Models](http://arxiv.org/abs/2210.03629v3) | Shunyu Yao, Jeffrey Zhao, Dian Yu, et al. | 2022-10-06 | `Docs:` [docs/integrations/providers/cohere](https://python.langchain.com/docs/integrations/providers/cohere), [docs/integrations/tools/ionic_shopping](https://python.langchain.com/docs/integrations/tools/ionic_shopping), `API:` [langchain...TrajectoryEvalChain](https://api.python.langchain.com/en/latest/evaluation/langchain.evaluation.agents.trajectory_eval_chain.TrajectoryEvalChain.html#langchain.evaluation.agents.trajectory_eval_chain.TrajectoryEvalChain), [langchain...create_react_agent](https://api.python.langchain.com/en/latest/agents/langchain.agents.react.agent.create_react_agent.html#langchain.agents.react.agent.create_react_agent)| `2209.10785v2` [Deep Lake: a Lakehouse for Deep Learning](http://arxiv.org/abs/2209.10785v2) | Sasun Hambardzumyan, Abhinav Tuli, Levon Ghukasyan, et al. | 2022-09-22 | `Docs:` [docs/integrations/providers/activeloop_deeplake](https://python.langchain.com/docs/integrations/providers/activeloop_deeplake)+| `2205.13147v4` [Matryoshka Representation Learning](http://arxiv.org/abs/2205.13147v4) | Aditya Kusupati, Gantavya Bhatt, Aniket Rege, et al. | 2022-05-26 | `Docs:` [docs/integrations/providers/snowflake](https://python.langchain.com/docs/integrations/providers/snowflake)| `2205.12654v1` [Bitext Mining Using Distilled Sentence Representations for Low-Resource Languages](http://arxiv.org/abs/2205.12654v1) | Kevin Heffernan, Onur Çelebi, Holger Schwenk | 2022-05-25 | `API:` [langchain_community...LaserEmbeddings](https://api.python.langchain.com/en/latest/embeddings/langchain_community.embeddings.laser.LaserEmbeddings.html#langchain_community.embeddings.laser.LaserEmbeddings)-| `2204.00498v1` [Evaluating the Text-to-SQL Capabilities of Large Language Models](http://arxiv.org/abs/2204.00498v1) | Nitarshan Rajkumar, Raymond Li, Dzmitry Bahdanau | 2022-03-15 | `API:` [langchain_community...SparkSQL](https://api.python.langchain.com/en/latest/utilities/langchain_community.utilities.spark_sql.SparkSQL.html#langchain_community.utilities.spark_sql.SparkSQL), [langchain_community...SQLDatabase](https://api.python.langchain.com/en/latest/utilities/langchain_community.utilities.sql_database.SQLDatabase.html#langchain_community.utilities.sql_database.SQLDatabase)-| `2202.00666v5` [Locally Typical Sampling](http://arxiv.org/abs/2202.00666v5) | Clara Meister, Tiago Pimentel, Gian Wiher, et al. | 2022-02-01 | `API:` [langchain_community...HuggingFaceEndpoint](https://api.python.langchain.com/en/latest/llms/langchain_community.llms.huggingface_endpoint.HuggingFaceEndpoint.html#langchain_community.llms.huggingface_endpoint.HuggingFaceEndpoint), [langchain_huggingface...HuggingFaceEndpoint](https://api.python.langchain.com/en/latest/llms/langchain_huggingface.llms.huggingface_endpoint.HuggingFaceEndpoint.html#langchain_huggingface.llms.huggingface_endpoint.HuggingFaceEndpoint), [langchain_community...HuggingFaceTextGenInference](https://api.python.langchain.com/en/latest/llms/langchain_community.llms.huggingface_text_gen_inference.HuggingFaceTextGenInference.html#langchain_community.llms.huggingface_text_gen_inference.HuggingFaceTextGenInference)+| `2204.00498v1` [Evaluating the Text-to-SQL Capabilities of Large Language Models](http://arxiv.org/abs/2204.00498v1) | Nitarshan Rajkumar, Raymond Li, Dzmitry Bahdanau | 2022-03-15 | `API:` [langchain_community...SQLDatabase](https://api.python.langchain.com/en/latest/utilities/langchain_community.utilities.sql_database.SQLDatabase.html#langchain_community.utilities.sql_database.SQLDatabase), [langchain_community...SparkSQL](https://api.python.langchain.com/en/latest/utilities/langchain_community.utilities.spark_sql.SparkSQL.html#langchain_community.utilities.spark_sql.SparkSQL)+| `2202.00666v5` [Locally Typical Sampling](http://arxiv.org/abs/2202.00666v5) | Clara Meister, Tiago Pimentel, Gian Wiher, et al. | 2022-02-01 | `API:` [langchain_huggingface...HuggingFaceEndpoint](https://api.python.langchain.com/en/latest/llms/langchain_huggingface.llms.huggingface_endpoint.HuggingFaceEndpoint.html#langchain_huggingface.llms.huggingface_endpoint.HuggingFaceEndpoint), [langchain_community...HuggingFaceEndpoint](https://api.python.langchain.com/en/latest/llms/langchain_community.llms.huggingface_endpoint.HuggingFaceEndpoint.html#langchain_community.llms.huggingface_endpoint.HuggingFaceEndpoint), [langchain_community...HuggingFaceTextGenInference](https://api.python.langchain.com/en/latest/llms/langchain_community.llms.huggingface_text_gen_inference.HuggingFaceTextGenInference.html#langchain_community.llms.huggingface_text_gen_inference.HuggingFaceTextGenInference)| `2103.00020v1` [Learning Transferable Visual Models From Natural Language Supervision](http://arxiv.org/abs/2103.00020v1) | Alec Radford, Jong Wook Kim, Chris Hallacy, et al. | 2021-02-26 | `API:` [langchain_experimental.open_clip](https://api.python.langchain.com/en/latest/experimental_api_reference.html#module-langchain_experimental.open_clip)-| `1909.05858v2` [CTRL: A Conditional Transformer Language Model for Controllable Generation](http://arxiv.org/abs/1909.05858v2) | Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, et al. | 2019-09-11 | `API:` [langchain_community...HuggingFaceEndpoint](https://api.python.langchain.com/en/latest/llms/langchain_community.llms.huggingface_endpoint.HuggingFaceEndpoint.html#langchain_community.llms.huggingface_endpoint.HuggingFaceEndpoint), [langchain_huggingface...HuggingFaceEndpoint](https://api.python.langchain.com/en/latest/llms/langchain_huggingface.llms.huggingface_endpoint.HuggingFaceEndpoint.html#langchain_huggingface.llms.huggingface_endpoint.HuggingFaceEndpoint), [langchain_community...HuggingFaceTextGenInference](https://api.python.langchain.com/en/latest/llms/langchain_community.llms.huggingface_text_gen_inference.HuggingFaceTextGenInference.html#langchain_community.llms.huggingface_text_gen_inference.HuggingFaceTextGenInference)-| `1908.10084v1` [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](http://arxiv.org/abs/1908.10084v1) | Nils Reimers, Iryna Gurevych | 2019-08-27 | `Docs:` [docs/integrations/text_embedding/sentence_transformers](https://python.langchain.com/docs/integrations/text_embedding/sentence_transformers)+| `1909.05858v2` [CTRL: A Conditional Transformer Language Model for Controllable Generation](http://arxiv.org/abs/1909.05858v2) | Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, et al. | 2019-09-11 | `API:` [langchain_huggingface...HuggingFaceEndpoint](https://api.python.langchain.com/en/latest/llms/langchain_huggingface.llms.huggingface_endpoint.HuggingFaceEndpoint.html#langchain_huggingface.llms.huggingface_endpoint.HuggingFaceEndpoint), [langchain_community...HuggingFaceEndpoint](https://api.python.langchain.com/en/latest/llms/langchain_community.llms.huggingface_endpoint.HuggingFaceEndpoint.html#langchain_community.llms.huggingface_endpoint.HuggingFaceEndpoint), [langchain_community...HuggingFaceTextGenInference](https://api.python.langchain.com/en/latest/llms/langchain_community.llms.huggingface_text_gen_inference.HuggingFaceTextGenInference.html#langchain_community.llms.huggingface_text_gen_inference.HuggingFaceTextGenInference)## Self-Discover: Large Language Models Self-Compose Reasoning Structures-- **arXiv id:** 2402.03620v1+- **arXiv id:** [2402.03620v1](http://arxiv.org/abs/2402.03620v1) **Published Date:** 2024-02-06- **Title:** Self-Discover: Large Language Models Self-Compose Reasoning Structures- **Authors:** Pei Zhou, Jay Pujara, Xiang Ren, et al.-- **Published Date:** 2024-02-06-- **URL:** http://arxiv.org/abs/2402.03620v1- **LangChain:**- **Cookbook:** [self-discover](https://github.com/langchain-ai/langchain/blob/master/cookbook/self-discover.ipynb)@@ -70,11 +71,9 @@ commonalities with human reasoning patterns.## RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval-- **arXiv id:** 2401.18059v1+- **arXiv id:** [2401.18059v1](http://arxiv.org/abs/2401.18059v1) **Published Date:** 2024-01-31- **Title:** RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval- **Authors:** Parth Sarthi, Salman Abdullah, Aditi Tuli, et al.-- **Published Date:** 2024-01-31-- **URL:** http://arxiv.org/abs/2401.18059v1- **LangChain:**- **Cookbook:** [RAPTOR](https://github.com/langchain-ai/langchain/blob/master/cookbook/RAPTOR.ipynb)@@ -96,11 +95,9 @@ benchmark by 20% in absolute accuracy.## Corrective Retrieval Augmented Generation-- **arXiv id:** 2401.15884v2+- **arXiv id:** [2401.15884v2](http://arxiv.org/abs/2401.15884v2) **Published Date:** 2024-01-29- **Title:** Corrective Retrieval Augmented Generation- **Authors:** Shi-Qi Yan, Jia-Chen Gu, Yun Zhu, et al.-- **Published Date:** 2024-01-29-- **URL:** http://arxiv.org/abs/2401.15884v2- **LangChain:**- **Cookbook:** [langgraph_crag](https://github.com/langchain-ai/langchain/blob/master/cookbook/langgraph_crag.ipynb)@@ -126,11 +123,9 @@ performance of RAG-based approaches.## Mixtral of Experts-- **arXiv id:** 2401.04088v1+- **arXiv id:** [2401.04088v1](http://arxiv.org/abs/2401.04088v1) **Published Date:** 2024-01-08- **Title:** Mixtral of Experts- **Authors:** Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, et al.-- **Published Date:** 2024-01-08-- **URL:** http://arxiv.org/abs/2401.04088v1- **LangChain:**- **Cookbook:** [together_ai](https://github.com/langchain-ai/langchain/blob/master/cookbook/together_ai.ipynb)@@ -152,11 +147,9 @@ the base and instruct models are released under the Apache 2.0 license.## Dense X Retrieval: What Retrieval Granularity Should We Use?-- **arXiv id:** 2312.06648v2+- **arXiv id:** [2312.06648v2](http://arxiv.org/abs/2312.06648v2) **Published Date:** 2023-12-11- **Title:** Dense X Retrieval: What Retrieval Granularity Should We Use?- **Authors:** Tong Chen, Hongwei Wang, Sihao Chen, et al.-- **Published Date:** 2023-12-11-- **URL:** http://arxiv.org/abs/2312.06648v2- **LangChain:**- **Template:** [propositional-retrieval](https://python.langchain.com/docs/templates/propositional-retrieval)@@ -181,11 +174,9 @@ information.## Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models-- **arXiv id:** 2311.09210v1+- **arXiv id:** [2311.09210v1](http://arxiv.org/abs/2311.09210v1) **Published Date:** 2023-11-15- **Title:** Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models- **Authors:** Wenhao Yu, Hongming Zhang, Xiaoman Pan, et al.-- **Published Date:** 2023-11-15-- **URL:** http://arxiv.org/abs/2311.09210v1- **LangChain:**- **Template:** [chain-of-note-wiki](https://python.langchain.com/docs/templates/chain-of-note-wiki)@@ -215,11 +206,9 @@ outside the pre-training knowledge scope.## Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection-- **arXiv id:** 2310.11511v1+- **arXiv id:** [2310.11511v1](http://arxiv.org/abs/2310.11511v1) **Published Date:** 2023-10-17- **Title:** Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection- **Authors:** Akari Asai, Zeqiu Wu, Yizhong Wang, et al.-- **Published Date:** 2023-10-17-- **URL:** http://arxiv.org/abs/2310.11511v1- **LangChain:**- **Cookbook:** [langgraph_self_rag](https://github.com/langchain-ai/langchain/blob/master/cookbook/langgraph_self_rag.ipynb)@@ -248,11 +237,9 @@ to these models.## Take a Step Back: Evoking Reasoning via Abstraction in Large Language Models-- **arXiv id:** 2310.06117v2+- **arXiv id:** [2310.06117v2](http://arxiv.org/abs/2310.06117v2) **Published Date:** 2023-10-09- **Title:** Take a Step Back: Evoking Reasoning via Abstraction in Large Language Models- **Authors:** Huaixiu Steven Zheng, Swaroop Mishra, Xinyun Chen, et al.-- **Published Date:** 2023-10-09-- **URL:** http://arxiv.org/abs/2310.06117v2- **LangChain:**- **Template:** [stepback-qa-prompting](https://python.langchain.com/docs/templates/stepback-qa-prompting)@@ -271,11 +258,9 @@ and 11% respectively, TimeQA by 27%, and MuSiQue by 7%.## Llama 2: Open Foundation and Fine-Tuned Chat Models-- **arXiv id:** 2307.09288v2+- **arXiv id:** [2307.09288v2](http://arxiv.org/abs/2307.09288v2) **Published Date:** 2023-07-18- **Title:** Llama 2: Open Foundation and Fine-Tuned Chat Models- **Authors:** Hugo Touvron, Louis Martin, Kevin Stone, et al.-- **Published Date:** 2023-07-18-- **URL:** http://arxiv.org/abs/2307.09288v2- **LangChain:**- **Cookbook:** [Semi_Structured_RAG](https://github.com/langchain-ai/langchain/blob/master/cookbook/Semi_Structured_RAG.ipynb)@@ -292,11 +277,9 @@ contribute to the responsible development of LLMs.## Query Rewriting for Retrieval-Augmented Large Language Models-- **arXiv id:** 2305.14283v3+- **arXiv id:** [2305.14283v3](http://arxiv.org/abs/2305.14283v3) **Published Date:** 2023-05-23- **Title:** Query Rewriting for Retrieval-Augmented Large Language Models- **Authors:** Xinbei Ma, Yeyun Gong, Pengcheng He, et al.-- **Published Date:** 2023-05-23-- **URL:** http://arxiv.org/abs/2305.14283v3- **LangChain:**- **Template:** [rewrite-retrieve-read](https://python.langchain.com/docs/templates/rewrite-retrieve-read)@@ -322,11 +305,9 @@ for retrieval-augmented LLM.## Large Language Model Guided Tree-of-Thought-- **arXiv id:** 2305.08291v1+- **arXiv id:** [2305.08291v1](http://arxiv.org/abs/2305.08291v1) **Published Date:** 2023-05-15- **Title:** Large Language Model Guided Tree-of-Thought- **Authors:** Jieyi Long-- **Published Date:** 2023-05-15-- **URL:** http://arxiv.org/abs/2305.08291v1- **LangChain:**- **API Reference:** [langchain_experimental.tot](https://api.python.langchain.com/en/latest/experimental_api_reference.html#module-langchain_experimental.tot)@@ -352,11 +333,9 @@ implementation of the ToT-based Sudoku solver is available on GitHub:## Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models-- **arXiv id:** 2305.04091v3+- **arXiv id:** [2305.04091v3](http://arxiv.org/abs/2305.04091v3) **Published Date:** 2023-05-06- **Title:** Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models- **Authors:** Lei Wang, Wanyu Xu, Yihuai Lan, et al.-- **Published Date:** 2023-05-06-- **URL:** http://arxiv.org/abs/2305.04091v3- **LangChain:**- **Cookbook:** [plan_and_execute_agent](https://github.com/langchain-ai/langchain/blob/master/cookbook/plan_and_execute_agent.ipynb)@@ -383,13 +362,35 @@ Prompting, and has comparable performance with 8-shot CoT prompting on the mathreasoning problem. The code can be found athttps://github.com/AGI-Edgerunners/Plan-and-Solve-Prompting.+## Zero-Shot Listwise Document Reranking with a Large Language Model++- **arXiv id:** [2305.02156v1](http://arxiv.org/abs/2305.02156v1) **Published Date:** 2023-05-03+- **Title:** Zero-Shot Listwise Document Reranking with a Large Language Model+- **Authors:** Xueguang Ma, Xinyu Zhang, Ronak Pradeep, et al.+- **LangChain:**++ - **API Reference:** [langchain...LLMListwiseRerank](https://api.python.langchain.com/en/latest/retrievers/langchain.retrievers.document_compressors.listwise_rerank.LLMListwiseRerank.html#langchain.retrievers.document_compressors.listwise_rerank.LLMListwiseRerank)++**Abstract:** Supervised ranking methods based on bi-encoder or cross-encoder architectures+have shown success in multi-stage text ranking tasks, but they require large+amounts of relevance judgments as training data. In this work, we propose+Listwise Reranker with a Large Language Model (LRL), which achieves strong+reranking effectiveness without using any task-specific training data.+Different from the existing pointwise ranking methods, where documents are+scored independently and ranked according to the scores, LRL directly generates+a reordered list of document identifiers given the candidate documents.+Experiments on three TREC web search datasets demonstrate that LRL not only+outperforms zero-shot pointwise methods when reranking first-stage retrieval+results, but can also act as a final-stage reranker to improve the top-ranked+results of a pointwise method for improved efficiency. Additionally, we apply+our approach to subsets of MIRACL, a recent multilingual retrieval dataset,+with results showing its potential to generalize across different languages.+## Visual Instruction Tuning-- **arXiv id:** 2304.08485v2+- **arXiv id:** [2304.08485v2](http://arxiv.org/abs/2304.08485v2) **Published Date:** 2023-04-17- **Title:** Visual Instruction Tuning- **Authors:** Haotian Liu, Chunyuan Li, Qingyang Wu, et al.-- **Published Date:** 2023-04-17-- **URL:** http://arxiv.org/abs/2304.08485v2- **LangChain:**- **Cookbook:** [Semi_structured_and_multi_modal_RAG](https://github.com/langchain-ai/langchain/blob/master/cookbook/Semi_structured_and_multi_modal_RAG.ipynb), [Semi_structured_multi_modal_RAG_LLaMA2](https://github.com/langchain-ai/langchain/blob/master/cookbook/Semi_structured_multi_modal_RAG_LLaMA2.ipynb)@@ -412,11 +413,9 @@ publicly available.## Generative Agents: Interactive Simulacra of Human Behavior-- **arXiv id:** 2304.03442v2+- **arXiv id:** [2304.03442v2](http://arxiv.org/abs/2304.03442v2) **Published Date:** 2023-04-07- **Title:** Generative Agents: Interactive Simulacra of Human Behavior- **Authors:** Joon Sung Park, Joseph C. O'Brien, Carrie J. Cai, et al.-- **Published Date:** 2023-04-07-- **URL:** http://arxiv.org/abs/2304.03442v2- **LangChain:**- **Cookbook:** [multiagent_bidding](https://github.com/langchain-ai/langchain/blob/master/cookbook/multiagent_bidding.ipynb), [generative_agents_interactive_simulacra_of_human_behavior](https://github.com/langchain-ai/langchain/blob/master/cookbook/generative_agents_interactive_simulacra_of_human_behavior.ipynb)@@ -448,11 +447,9 @@ interaction patterns for enabling believable simulations of human behavior.## CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society-- **arXiv id:** 2303.17760v2+- **arXiv id:** [2303.17760v2](http://arxiv.org/abs/2303.17760v2) **Published Date:** 2023-03-31- **Title:** CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society- **Authors:** Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, et al.-- **Published Date:** 2023-03-31-- **URL:** http://arxiv.org/abs/2303.17760v2- **LangChain:**- **Cookbook:** [camel_role_playing](https://github.com/langchain-ai/langchain/blob/master/cookbook/camel_role_playing.ipynb)@@ -478,11 +475,9 @@ agents and beyond: https://github.com/camel-ai/camel.## HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face-- **arXiv id:** 2303.17580v4+- **arXiv id:** [2303.17580v4](http://arxiv.org/abs/2303.17580v4) **Published Date:** 2023-03-30- **Title:** HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face- **Authors:** Yongliang Shen, Kaitao Song, Xu Tan, et al.-- **Published Date:** 2023-03-30-- **URL:** http://arxiv.org/abs/2303.17580v4- **LangChain:**- **API Reference:** [langchain_experimental.autonomous_agents](https://api.python.langchain.com/en/latest/experimental_api_reference.html#module-langchain_experimental.autonomous_agents)@@ -508,40 +503,14 @@ modalities and domains and achieve impressive results in language, vision,speech, and other challenging tasks, which paves a new way towards therealization of artificial general intelligence.-## GPT-4 Technical Report--- **arXiv id:** 2303.08774v6-- **Title:** GPT-4 Technical Report-- **Authors:** OpenAI, Josh Achiam, Steven Adler, et al.-- **Published Date:** 2023-03-15-- **URL:** http://arxiv.org/abs/2303.08774v6-- **LangChain:**-- - **Documentation:** [docs/integrations/vectorstores/mongodb_atlas](https://python.langchain.com/docs/integrations/vectorstores/mongodb_atlas)--**Abstract:** We report the development of GPT-4, a large-scale, multimodal model which can-accept image and text inputs and produce text outputs. While less capable than-humans in many real-world scenarios, GPT-4 exhibits human-level performance on-various professional and academic benchmarks, including passing a simulated bar-exam with a score around the top 10% of test takers. GPT-4 is a-Transformer-based model pre-trained to predict the next token in a document.-The post-training alignment process results in improved performance on measures-of factuality and adherence to desired behavior. A core component of this-project was developing infrastructure and optimization methods that behave-predictably across a wide range of scales. This allowed us to accurately-predict some aspects of GPT-4's performance based on models trained with no-more than 1/1,000th the compute of GPT-4.-## A Watermark for Large Language Models-- **arXiv id:** 2301.10226v4+- **arXiv id:** [2301.10226v4](http://arxiv.org/abs/2301.10226v4) **Published Date:** 2023-01-24- **Title:** A Watermark for Large Language Models- **Authors:** John Kirchenbauer, Jonas Geiping, Yuxin Wen, et al.-- **Published Date:** 2023-01-24-- **URL:** http://arxiv.org/abs/2301.10226v4- **LangChain:**- - **API Reference:** [langchain_community...HuggingFaceEndpoint](https://api.python.langchain.com/en/latest/llms/langchain_community.llms.huggingface_endpoint.HuggingFaceEndpoint.html#langchain_community.llms.huggingface_endpoint.HuggingFaceEndpoint), [langchain_huggingface...HuggingFaceEndpoint](https://api.python.langchain.com/en/latest/llms/langchain_huggingface.llms.huggingface_endpoint.HuggingFaceEndpoint.html#langchain_huggingface.llms.huggingface_endpoint.HuggingFaceEndpoint), [langchain_community...OCIModelDeploymentTGI](https://api.python.langchain.com/en/latest/llms/langchain_community.llms.oci_data_science_model_deployment_endpoint.OCIModelDeploymentTGI.html#langchain_community.llms.oci_data_science_model_deployment_endpoint.OCIModelDeploymentTGI), [langchain_community...HuggingFaceTextGenInference](https://api.python.langchain.com/en/latest/llms/langchain_community.llms.huggingface_text_gen_inference.HuggingFaceTextGenInference.html#langchain_community.llms.huggingface_text_gen_inference.HuggingFaceTextGenInference)+ - **API Reference:** [langchain_community...OCIModelDeploymentTGI](https://api.python.langchain.com/en/latest/llms/langchain_community.llms.oci_data_science_model_deployment_endpoint.OCIModelDeploymentTGI.html#langchain_community.llms.oci_data_science_model_deployment_endpoint.OCIModelDeploymentTGI), [langchain_huggingface...HuggingFaceEndpoint](https://api.python.langchain.com/en/latest/llms/langchain_huggingface.llms.huggingface_endpoint.HuggingFaceEndpoint.html#langchain_huggingface.llms.huggingface_endpoint.HuggingFaceEndpoint), [langchain_community...HuggingFaceEndpoint](https://api.python.langchain.com/en/latest/llms/langchain_community.llms.huggingface_endpoint.HuggingFaceEndpoint.html#langchain_community.llms.huggingface_endpoint.HuggingFaceEndpoint), [langchain_community...HuggingFaceTextGenInference](https://api.python.langchain.com/en/latest/llms/langchain_community.llms.huggingface_text_gen_inference.HuggingFaceTextGenInference.html#langchain_community.llms.huggingface_text_gen_inference.HuggingFaceTextGenInference)**Abstract:** Potential harms of large language models can be mitigated by watermarkingmodel output, i.e., embedding signals into generated text that are invisible to@@ -559,11 +528,9 @@ family, and discuss robustness and security.## Precise Zero-Shot Dense Retrieval without Relevance Labels-- **arXiv id:** 2212.10496v1+- **arXiv id:** [2212.10496v1](http://arxiv.org/abs/2212.10496v1) **Published Date:** 2022-12-20- **Title:** Precise Zero-Shot Dense Retrieval without Relevance Labels- **Authors:** Luyu Gao, Xueguang Ma, Jimmy Lin, et al.-- **Published Date:** 2022-12-20-- **URL:** http://arxiv.org/abs/2212.10496v1- **LangChain:**- **API Reference:** [langchain...HypotheticalDocumentEmbedder](https://api.python.langchain.com/en/latest/chains/langchain.chains.hyde.base.HypotheticalDocumentEmbedder.html#langchain.chains.hyde.base.HypotheticalDocumentEmbedder)@@ -590,11 +557,9 @@ search, QA, fact verification) and languages~(e.g. sw, ko, ja).## Robust and Explainable Identification of Logical Fallacies in Natural Language Arguments-- **arXiv id:** 2212.07425v3+- **arXiv id:** [2212.07425v3](http://arxiv.org/abs/2212.07425v3) **Published Date:** 2022-12-12- **Title:** Robust and Explainable Identification of Logical Fallacies in Natural Language Arguments- **Authors:** Zhivar Sourati, Vishnu Priya Prasanna Venkatesh, Darshan Deshpande, et al.-- **Published Date:** 2022-12-12-- **URL:** http://arxiv.org/abs/2212.07425v3- **LangChain:**- **API Reference:** [langchain_experimental.fallacy_removal](https://api.python.langchain.com/en/latest/experimental_api_reference.html#module-langchain_experimental.fallacy_removal)@@ -623,11 +588,9 @@ further work on logical fallacy identification.## Complementary Explanations for Effective In-Context Learning-- **arXiv id:** 2211.13892v2+- **arXiv id:** [2211.13892v2](http://arxiv.org/abs/2211.13892v2) **Published Date:** 2022-11-25- **Title:** Complementary Explanations for Effective In-Context Learning- **Authors:** Xi Ye, Srinivasan Iyer, Asli Celikyilmaz, et al.-- **Published Date:** 2022-11-25-- **URL:** http://arxiv.org/abs/2211.13892v2- **LangChain:**- **API Reference:** [langchain_core...MaxMarginalRelevanceExampleSelector](https://api.python.langchain.com/en/latest/example_selectors/langchain_core.example_selectors.semantic_similarity.MaxMarginalRelevanceExampleSelector.html#langchain_core.example_selectors.semantic_similarity.MaxMarginalRelevanceExampleSelector)@@ -651,14 +614,12 @@ performance across three real-world tasks on multiple LLMs.## PAL: Program-aided Language Models-- **arXiv id:** 2211.10435v2+- **arXiv id:** [2211.10435v2](http://arxiv.org/abs/2211.10435v2) **Published Date:** 2022-11-18- **Title:** PAL: Program-aided Language Models- **Authors:** Luyu Gao, Aman Madaan, Shuyan Zhou, et al.-- **Published Date:** 2022-11-18-- **URL:** http://arxiv.org/abs/2211.10435v2- **LangChain:**- - **API Reference:** [langchain_experimental...PALChain](https://api.python.langchain.com/en/latest/pal_chain/langchain_experimental.pal_chain.base.PALChain.html#langchain_experimental.pal_chain.base.PALChain), [langchain_experimental.pal_chain](https://api.python.langchain.com/en/latest/experimental_api_reference.html#module-langchain_experimental.pal_chain)+ - **API Reference:** [langchain_experimental.pal_chain](https://api.python.langchain.com/en/latest/experimental_api_reference.html#module-langchain_experimental.pal_chain), [langchain_experimental...PALChain](https://api.python.langchain.com/en/latest/pal_chain/langchain_experimental.pal_chain.base.PALChain.html#langchain_experimental.pal_chain.base.PALChain)- **Cookbook:** [program_aided_language_model](https://github.com/langchain-ai/langchain/blob/master/cookbook/program_aided_language_model.ipynb)**Abstract:** Large language models (LLMs) have recently demonstrated an impressive ability@@ -686,15 +647,13 @@ publicly available at http://reasonwithpal.com/ .## ReAct: Synergizing Reasoning and Acting in Language Models-- **arXiv id:** 2210.03629v3+- **arXiv id:** [2210.03629v3](http://arxiv.org/abs/2210.03629v3) **Published Date:** 2022-10-06- **Title:** ReAct: Synergizing Reasoning and Acting in Language Models- **Authors:** Shunyu Yao, Jeffrey Zhao, Dian Yu, et al.-- **Published Date:** 2022-10-06-- **URL:** http://arxiv.org/abs/2210.03629v3- **LangChain:**- - **Documentation:** [docs/integrations/providers/cohere](https://python.langchain.com/docs/integrations/providers/cohere), [docs/integrations/chat/huggingface](https://python.langchain.com/docs/integrations/chat/huggingface), [docs/integrations/tools/ionic_shopping](https://python.langchain.com/docs/integrations/tools/ionic_shopping)- - **API Reference:** [langchain...create_react_agent](https://api.python.langchain.com/en/latest/agents/langchain.agents.react.agent.create_react_agent.html#langchain.agents.react.agent.create_react_agent), [langchain...TrajectoryEvalChain](https://api.python.langchain.com/en/latest/evaluation/langchain.evaluation.agents.trajectory_eval_chain.TrajectoryEvalChain.html#langchain.evaluation.agents.trajectory_eval_chain.TrajectoryEvalChain)+ - **Documentation:** [docs/integrations/providers/cohere](https://python.langchain.com/docs/integrations/providers/cohere), [docs/integrations/tools/ionic_shopping](https://python.langchain.com/docs/integrations/tools/ionic_shopping)+ - **API Reference:** [langchain...TrajectoryEvalChain](https://api.python.langchain.com/en/latest/evaluation/langchain.evaluation.agents.trajectory_eval_chain.TrajectoryEvalChain.html#langchain.evaluation.agents.trajectory_eval_chain.TrajectoryEvalChain), [langchain...create_react_agent](https://api.python.langchain.com/en/latest/agents/langchain.agents.react.agent.create_react_agent.html#langchain.agents.react.agent.create_react_agent)**Abstract:** While large language models (LLMs) have demonstrated impressive capabilitiesacross tasks in language understanding and interactive decision making, their@@ -721,11 +680,9 @@ Project site with code: https://react-lm.github.io## Deep Lake: a Lakehouse for Deep Learning-- **arXiv id:** 2209.10785v2+- **arXiv id:** [2209.10785v2](http://arxiv.org/abs/2209.10785v2) **Published Date:** 2022-09-22- **Title:** Deep Lake: a Lakehouse for Deep Learning- **Authors:** Sasun Hambardzumyan, Abhinav Tuli, Levon Ghukasyan, et al.-- **Published Date:** 2022-09-22-- **URL:** http://arxiv.org/abs/2209.10785v2- **LangChain:**- **Documentation:** [docs/integrations/providers/activeloop_deeplake](https://python.langchain.com/docs/integrations/providers/activeloop_deeplake)@@ -747,13 +704,43 @@ visualization engine, or (c) deep learning frameworks without sacrificing GPUutilization. Datasets stored in Deep Lake can be accessed from PyTorch,TensorFlow, JAX, and integrate with numerous MLOps tools.+## Matryoshka Representation Learning++- **arXiv id:** [2205.13147v4](http://arxiv.org/abs/2205.13147v4) **Published Date:** 2022-05-26+- **Title:** Matryoshka Representation Learning+- **Authors:** Aditya Kusupati, Gantavya Bhatt, Aniket Rege, et al.+- **LangChain:**++ - **Documentation:** [docs/integrations/providers/snowflake](https://python.langchain.com/docs/integrations/providers/snowflake)++**Abstract:** Learned representations are a central component in modern ML systems, serving+a multitude of downstream tasks. When training such representations, it is+often the case that computational and statistical constraints for each+downstream task are unknown. In this context rigid, fixed capacity+representations can be either over or under-accommodating to the task at hand.+This leads us to ask: can we design a flexible representation that can adapt to+multiple downstream tasks with varying computational resources? Our main+contribution is Matryoshka Representation Learning (MRL) which encodes+information at different granularities and allows a single embedding to adapt+to the computational constraints of downstream tasks. MRL minimally modifies+existing representation learning pipelines and imposes no additional cost+during inference and deployment. MRL learns coarse-to-fine representations that+are at least as accurate and rich as independently trained low-dimensional+representations. The flexibility within the learned Matryoshka Representations+offer: (a) up to 14x smaller embedding size for ImageNet-1K classification at+the same level of accuracy; (b) up to 14x real-world speed-ups for large-scale+retrieval on ImageNet-1K and 4K; and (c) up to 2% accuracy improvements for+long-tail few-shot classification, all while being as robust as the original+representations. Finally, we show that MRL extends seamlessly to web-scale+datasets (ImageNet, JFT) across various modalities -- vision (ViT, ResNet),+vision + language (ALIGN) and language (BERT). MRL code and pretrained models+are open-sourced at https://github.com/RAIVNLab/MRL.+## Bitext Mining Using Distilled Sentence Representations for Low-Resource Languages-- **arXiv id:** 2205.12654v1+- **arXiv id:** [2205.12654v1](http://arxiv.org/abs/2205.12654v1) **Published Date:** 2022-05-25- **Title:** Bitext Mining Using Distilled Sentence Representations for Low-Resource Languages- **Authors:** Kevin Heffernan, Onur Çelebi, Holger Schwenk-- **Published Date:** 2022-05-25-- **URL:** http://arxiv.org/abs/2205.12654v1- **LangChain:**- **API Reference:** [langchain_community...LaserEmbeddings](https://api.python.langchain.com/en/latest/embeddings/langchain_community.embeddings.laser.LaserEmbeddings.html#langchain_community.embeddings.laser.LaserEmbeddings)@@ -778,14 +765,12 @@ encoders, mine bitexts, and validate the bitexts by training NMT systems.## Evaluating the Text-to-SQL Capabilities of Large Language Models-- **arXiv id:** 2204.00498v1+- **arXiv id:** [2204.00498v1](http://arxiv.org/abs/2204.00498v1) **Published Date:** 2022-03-15- **Title:** Evaluating the Text-to-SQL Capabilities of Large Language Models- **Authors:** Nitarshan Rajkumar, Raymond Li, Dzmitry Bahdanau-- **Published Date:** 2022-03-15-- **URL:** http://arxiv.org/abs/2204.00498v1- **LangChain:**- - **API Reference:** [langchain_community...SparkSQL](https://api.python.langchain.com/en/latest/utilities/langchain_community.utilities.spark_sql.SparkSQL.html#langchain_community.utilities.spark_sql.SparkSQL), [langchain_community...SQLDatabase](https://api.python.langchain.com/en/latest/utilities/langchain_community.utilities.sql_database.SQLDatabase.html#langchain_community.utilities.sql_database.SQLDatabase)+ - **API Reference:** [langchain_community...SQLDatabase](https://api.python.langchain.com/en/latest/utilities/langchain_community.utilities.sql_database.SQLDatabase.html#langchain_community.utilities.sql_database.SQLDatabase), [langchain_community...SparkSQL](https://api.python.langchain.com/en/latest/utilities/langchain_community.utilities.spark_sql.SparkSQL.html#langchain_community.utilities.spark_sql.SparkSQL)**Abstract:** We perform an empirical evaluation of Text-to-SQL capabilities of the Codexlanguage model. We find that, without any finetuning, Codex is a strong@@ -797,14 +782,12 @@ few-shot examples.## Locally Typical Sampling-- **arXiv id:** 2202.00666v5+- **arXiv id:** [2202.00666v5](http://arxiv.org/abs/2202.00666v5) **Published Date:** 2022-02-01- **Title:** Locally Typical Sampling- **Authors:** Clara Meister, Tiago Pimentel, Gian Wiher, et al.-- **Published Date:** 2022-02-01-- **URL:** http://arxiv.org/abs/2202.00666v5- **LangChain:**- - **API Reference:** [langchain_community...HuggingFaceEndpoint](https://api.python.langchain.com/en/latest/llms/langchain_community.llms.huggingface_endpoint.HuggingFaceEndpoint.html#langchain_community.llms.huggingface_endpoint.HuggingFaceEndpoint), [langchain_huggingface...HuggingFaceEndpoint](https://api.python.langchain.com/en/latest/llms/langchain_huggingface.llms.huggingface_endpoint.HuggingFaceEndpoint.html#langchain_huggingface.llms.huggingface_endpoint.HuggingFaceEndpoint), [langchain_community...HuggingFaceTextGenInference](https://api.python.langchain.com/en/latest/llms/langchain_community.llms.huggingface_text_gen_inference.HuggingFaceTextGenInference.html#langchain_community.llms.huggingface_text_gen_inference.HuggingFaceTextGenInference)+ - **API Reference:** [langchain_huggingface...HuggingFaceEndpoint](https://api.python.langchain.com/en/latest/llms/langchain_huggingface.llms.huggingface_endpoint.HuggingFaceEndpoint.html#langchain_huggingface.llms.huggingface_endpoint.HuggingFaceEndpoint), [langchain_community...HuggingFaceEndpoint](https://api.python.langchain.com/en/latest/llms/langchain_community.llms.huggingface_endpoint.HuggingFaceEndpoint.html#langchain_community.llms.huggingface_endpoint.HuggingFaceEndpoint), [langchain_community...HuggingFaceTextGenInference](https://api.python.langchain.com/en/latest/llms/langchain_community.llms.huggingface_text_gen_inference.HuggingFaceTextGenInference.html#langchain_community.llms.huggingface_text_gen_inference.HuggingFaceTextGenInference)**Abstract:** Today's probabilistic language generators fall short when it comes toproducing coherent and fluent text despite the fact that the underlying models@@ -829,11 +812,9 @@ reducing degenerate repetitions.## Learning Transferable Visual Models From Natural Language Supervision-- **arXiv id:** 2103.00020v1+- **arXiv id:** [2103.00020v1](http://arxiv.org/abs/2103.00020v1) **Published Date:** 2021-02-26- **Title:** Learning Transferable Visual Models From Natural Language Supervision- **Authors:** Alec Radford, Jong Wook Kim, Chris Hallacy, et al.-- **Published Date:** 2021-02-26-- **URL:** http://arxiv.org/abs/2103.00020v1- **LangChain:**- **API Reference:** [langchain_experimental.open_clip](https://api.python.langchain.com/en/latest/experimental_api_reference.html#module-langchain_experimental.open_clip)@@ -861,14 +842,12 @@ https://github.com/OpenAI/CLIP.## CTRL: A Conditional Transformer Language Model for Controllable Generation-- **arXiv id:** 1909.05858v2+- **arXiv id:** [1909.05858v2](http://arxiv.org/abs/1909.05858v2) **Published Date:** 2019-09-11- **Title:** CTRL: A Conditional Transformer Language Model for Controllable Generation- **Authors:** Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, et al.-- **Published Date:** 2019-09-11-- **URL:** http://arxiv.org/abs/1909.05858v2- **LangChain:**- - **API Reference:** [langchain_community...HuggingFaceEndpoint](https://api.python.langchain.com/en/latest/llms/langchain_community.llms.huggingface_endpoint.HuggingFaceEndpoint.html#langchain_community.llms.huggingface_endpoint.HuggingFaceEndpoint), [langchain_huggingface...HuggingFaceEndpoint](https://api.python.langchain.com/en/latest/llms/langchain_huggingface.llms.huggingface_endpoint.HuggingFaceEndpoint.html#langchain_huggingface.llms.huggingface_endpoint.HuggingFaceEndpoint), [langchain_community...HuggingFaceTextGenInference](https://api.python.langchain.com/en/latest/llms/langchain_community.llms.huggingface_text_gen_inference.HuggingFaceTextGenInference.html#langchain_community.llms.huggingface_text_gen_inference.HuggingFaceTextGenInference)+ - **API Reference:** [langchain_huggingface...HuggingFaceEndpoint](https://api.python.langchain.com/en/latest/llms/langchain_huggingface.llms.huggingface_endpoint.HuggingFaceEndpoint.html#langchain_huggingface.llms.huggingface_endpoint.HuggingFaceEndpoint), [langchain_community...HuggingFaceEndpoint](https://api.python.langchain.com/en/latest/llms/langchain_community.llms.huggingface_endpoint.HuggingFaceEndpoint.html#langchain_community.llms.huggingface_endpoint.HuggingFaceEndpoint), [langchain_community...HuggingFaceTextGenInference](https://api.python.langchain.com/en/latest/llms/langchain_community.llms.huggingface_text_gen_inference.HuggingFaceTextGenInference.html#langchain_community.llms.huggingface_text_gen_inference.HuggingFaceTextGenInference)**Abstract:** Large-scale language models show promising text generation capabilities, butusers cannot easily control particular aspects of the generated text. We@@ -881,32 +860,4 @@ codes also allow CTRL to predict which parts of the training data are mostlikely given a sequence. This provides a potential method for analyzing largeamounts of data via model-based source attribution. We have released multiplefull-sized, pretrained versions of CTRL at https://github.com/salesforce/ctrl.--## Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks--- **arXiv id:** 1908.10084v1-- **Title:** Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks-- **Authors:** Nils Reimers, Iryna Gurevych-- **Published Date:** 2019-08-27-- **URL:** http://arxiv.org/abs/1908.10084v1-- **LangChain:**-- - **Documentation:** [docs/integrations/text_embedding/sentence_transformers](https://python.langchain.com/docs/integrations/text_embedding/sentence_transformers)--**Abstract:** BERT (Devlin et al., 2018) and RoBERTa (Liu et al., 2019) has set a new-state-of-the-art performance on sentence-pair regression tasks like semantic-textual similarity (STS). However, it requires that both sentences are fed into-the network, which causes a massive computational overhead: Finding the most-similar pair in a collection of 10,000 sentences requires about 50 million-inference computations (~65 hours) with BERT. The construction of BERT makes it-unsuitable for semantic similarity search as well as for unsupervised tasks-like clustering.- In this publication, we present Sentence-BERT (SBERT), a modification of the-pretrained BERT network that use siamese and triplet network structures to-derive semantically meaningful sentence embeddings that can be compared using-cosine-similarity. This reduces the effort for finding the most similar pair-from 65 hours with BERT / RoBERTa to about 5 seconds with SBERT, while-maintaining the accuracy from BERT.- We evaluate SBERT and SRoBERTa on common STS tasks and transfer learning-tasks, where it outperforms other state-of-the-art sentence embeddings methods.\ No newline at end of filediff --git a/docs/scripts/arxiv_references.py b/docs/scripts/arxiv_references.pyindex 97bcc1148a..d498fcf08d 100644--- a/docs/scripts/arxiv_references.py+++ b/docs/scripts/arxiv_references.py@@ -522,8 +522,11 @@ LangChain implements the latest research in the field of Natural Language ProcesThis page contains `arXiv` papers referenced in the LangChain Documentation, API Reference,Templates, and Cookbooks.-From the opposite direction, scientists use LangChain in research and reference LangChain in the research papers.-Here you find [such papers](https://arxiv.org/search/?query=langchain&searchtype=all&source=header).+From the opposite direction, scientists use `LangChain` in research and reference it in the research papers.+Here you find papers that reference:+- [LangChain](https://arxiv.org/search/?query=langchain&searchtype=all&source=header)+- [LangGraph](https://arxiv.org/search/?query=langgraph&searchtype=all&source=header)+- [LangSmith](https://arxiv.org/search/?query=langsmith&searchtype=all&source=header)## Summary@@ -604,11 +607,9 @@ Here you find [such papers](https://arxiv.org/search/?query=langchain&searchtypef"""## {paper.title}-- **arXiv id:** {paper.arxiv_id}+- **arXiv id:** [{paper.arxiv_id}]({paper.url}) **Published Date:** {paper.published_date}- **Title:** {paper.title}- **Authors:** {', '.join(paper.authors)}-- **Published Date:** {paper.published_date}-- **URL:** {paper.url}- **LangChain:**{refs}