The agent engineering platform.
Python agentsaiai-agentsanthropicchatgptdeepagentsenterpriseframeworkgeminigenerative-ailangchainlanggraphllmmultiagentopen-sourceopenaipydanticpythonragtypescript
docs: replace initialize_agent with create_react_agent in searchapi.ipynb (#31184)
Michael Li65fbbb024955
diff --git a/docs/docs/integrations/tools/searchapi.ipynb b/docs/docs/integrations/tools/searchapi.ipynbindex c086b4d18d..cf18ca112a 100644--- a/docs/docs/integrations/tools/searchapi.ipynb+++ b/docs/docs/integrations/tools/searchapi.ipynb@@ -12,7 +12,7 @@},{"cell_type": "code",- "execution_count": 12,+ "execution_count": 1,"id": "70871a99-ffee-47d7-8e02-82eb99971f28","metadata": {},"outputs": [],@@ -51,7 +51,7 @@{"data": {"text/plain": [- "'Barack Hussein Obama II'"+ "'Barack Obama Full name: Barack Hussein Obama II'"]},"execution_count": 4,@@ -73,7 +73,7 @@},{"cell_type": "code",- "execution_count": 13,+ "execution_count": 5,"id": "17a9b1ad-6e84-4949-8ebd-8c52f6b296e3","metadata": {},"outputs": [],@@ -83,48 +83,11 @@},{"cell_type": "code",- "execution_count": 7,+ "execution_count": 6,"id": "cf8970a5-00e1-46bd-ba53-6a974eebbc10","metadata": {},- "outputs": [- {- "name": "stdout",- "output_type": "stream",- "text": [- "\n",- "\n",- "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",- "\u001b[32;1m\u001b[1;3m Yes.\n",- "Follow up: How old was Plato when he died?\u001b[0m\n",- "Intermediate answer: \u001b[36;1m\u001b[1;3meighty\u001b[0m\n",- "\u001b[32;1m\u001b[1;3mFollow up: How old was Socrates when he died?\u001b[0m\n",- "Intermediate answer: \u001b[36;1m\u001b[1;3m| Socrates | \n",- "| -------- | \n",- "| Born | c. 470 BC Deme Alopece, Athens | \n",- "| Died | 399 BC (aged approximately 71) Athens | \n",- "| Cause of death | Execution by forced suicide by poisoning | \n",- "| Spouse(s) | Xanthippe, Myrto | \n",- "\u001b[0m\n",- "\u001b[32;1m\u001b[1;3mFollow up: How old was Aristotle when he died?\u001b[0m\n",- "Intermediate answer: \u001b[36;1m\u001b[1;3m62 years\u001b[0m\n",- "\u001b[32;1m\u001b[1;3mSo the final answer is: Plato\u001b[0m\n",- "\n",- "\u001b[1m> Finished chain.\u001b[0m\n"- ]- },- {- "data": {- "text/plain": [- "'Plato'"- ]- },- "execution_count": 7,- "metadata": {},- "output_type": "execute_result"- }- ],+ "outputs": [],"source": [- "from langchain.agents import AgentType, initialize_agent\n","from langchain_community.utilities import SearchApiAPIWrapper\n","from langchain_core.tools import Tool\n","from langchain_openai import OpenAI\n",@@ -133,16 +96,88 @@"search = SearchApiAPIWrapper()\n","tools = [\n"," Tool(\n",- " name=\"Intermediate Answer\",\n",+ " name=\"intermediate_answer\",\n"," func=search.run,\n"," description=\"useful for when you need to ask with search\",\n"," )\n",- "]\n",+ "]"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 7,+ "id": "4198dda8-b7a9-4ae9-bcb6-b95e2c7681b9",+ "metadata": {},+ "outputs": [],+ "source": [+ "from langgraph.prebuilt import create_react_agent\n","\n",- "self_ask_with_search = initialize_agent(\n",- " tools, llm, agent=AgentType.SELF_ASK_WITH_SEARCH, verbose=True\n",- ")\n",- "self_ask_with_search.run(\"Who lived longer: Plato, Socrates, or Aristotle?\")"+ "agent = create_react_agent(\"openai:gpt-4.1-mini\", tools)"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 8,+ "id": "c24ad140-d41f-4e99-a42f-11371c3897b5",+ "metadata": {},+ "outputs": [+ {+ "name": "stdout",+ "output_type": "stream",+ "text": [+ "================================\u001b[1m Human Message \u001b[0m=================================\n",+ "\n",+ "Who lived longer: Plato, Socrates, or Aristotle?\n",+ "==================================\u001b[1m Ai Message \u001b[0m==================================\n",+ "Tool Calls:\n",+ " intermediate_answer (call_Q0JquDV3SWfnn3rkwJkJaffG)\n",+ " Call ID: call_Q0JquDV3SWfnn3rkwJkJaffG\n",+ " Args:\n",+ " __arg1: Lifespan of Plato\n",+ " intermediate_answer (call_j9rXzVlrCcGc8HOFnKUH6j5E)\n",+ " Call ID: call_j9rXzVlrCcGc8HOFnKUH6j5E\n",+ " Args:\n",+ " __arg1: Lifespan of Socrates\n",+ " intermediate_answer (call_IBQT2qn5PzDE6q0ZyfPdhRaX)\n",+ " Call ID: call_IBQT2qn5PzDE6q0ZyfPdhRaX\n",+ " Args:\n",+ " __arg1: Lifespan of Aristotle\n",+ "=================================\u001b[1m Tool Message \u001b[0m=================================\n",+ "Name: intermediate_answer\n",+ "\n",+ "384–322 BC was an Ancient Greek philosopher and polymath. His writings cover a broad range of subjects spanning the natural sciences, philosophy, linguistics, ...\n",+ "The Greek philosopher Aristotle (384-322 B.C.) made significant and lasting contributions to nearly every aspect of human knowledge, ...\n",+ "Aristotle's lifespan (384 - 322) (jan 1, 384 BC – jan 1, 322 BC). Added to timeline: Political Philosophy timeline. ByEdoardo. 25 Aug 2020.\n",+ "Aristotle was one of the greatest philosophers and scientists the world has ever seen. He was born in 384 bc at Stagirus, a Greek seaport on the coast of Thrace ...\n",+ "393–c. 370 bce), king of Macedonia and grandfather of Alexander the Great (reigned 336–323 bce). After his father's death in 367, Aristotle ...\n",+ "It is difficult to rule out that possibility decisively, since little is known about the period of Aristotle's life from 341–335. He evidently ...\n",+ "Lifespan: c. 384 B.C. to 322 B.C.; Contributions: Considered one of the greatest thinkers in various fields including politics, psychology, and ...\n",+ "Aristotle (Greek: Ἀριστοτέλης Aristotélēs, pronounced [aristotélɛːs]) lived 384–322 BC.\n",+ "Aristotle (384 B.C.E.—322 B.C.E.). Aristotle is a towering figure in ancient Greek philosophy, who made important contributions to logic, criticism, ...\n",+ "Aristotle. Born: 384 BC in Stagirus, Macedonia, Greece Died: 322 BC in Chalcis, Euboea, Greece. Aristotle was not primarily a mathematician but made ...\n",+ "==================================\u001b[1m Ai Message \u001b[0m==================================\n",+ "\n",+ "Based on the information:\n",+ "\n",+ "- Plato reportedly lived to be around eighty or eighty-one years old.\n",+ "- Socrates' exact lifespan is not directly stated here, but he is known historically to have lived approximately from 470 BC to 399 BC, making him around 71 years old.\n",+ "- Aristotle lived from 384 BC to 322 BC, which means he was about 62 years old.\n",+ "\n",+ "Therefore, Plato lived longer than both Socrates and Aristotle.\n"+ ]+ }+ ],+ "source": [+ "input_message = {\n",+ " \"role\": \"user\",\n",+ " \"content\": \"Who lived longer: Plato, Socrates, or Aristotle?\",\n",+ "}\n",+ "\n",+ "for step in agent.stream(\n",+ " {\"messages\": [input_message]},\n",+ " stream_mode=\"values\",\n",+ "):\n",+ " step[\"messages\"][-1].pretty_print()"]},{@@ -157,7 +192,7 @@},{"cell_type": "code",- "execution_count": 8,+ "execution_count": 9,"id": "6d0b4411-780a-4dcf-91b6-f3544e31e532","metadata": {},"outputs": [],@@ -167,17 +202,17 @@},{"cell_type": "code",- "execution_count": 9,+ "execution_count": 10,"id": "34e79449-6b33-4b45-9306-7e3dab1b8599","metadata": {},"outputs": [{"data": {"text/plain": [- "'Azure AI Engineer Be an XpanderCandidatar-meCandidatar-meCandidatar-me\\n\\nShare:\\n\\nAzure AI Engineer\\n\\nA área Digital Xperience da Xpand IT é uma equipa tecnológica de rápido crescimento que se concentra em tecnologias Microsoft e Mobile. A sua principal missão é fornecer soluções de software de alta qualidade que atendam às necessidades do utilizador final, num mundo tecnológico continuamente exigente e em ritmo acelerado, proporcionando a melhor experiência em termos de personalização, performance'"+ "'No good search result found'"]},- "execution_count": 9,+ "execution_count": 10,"metadata": {},"output_type": "execute_result"}@@ -196,7 +231,7 @@},{"cell_type": "code",- "execution_count": 10,+ "execution_count": 11,"id": "b16b7cd9-f0fe-4030-a36b-bbb52b19da18","metadata": {},"outputs": [],@@ -206,7 +241,7 @@},{"cell_type": "code",- "execution_count": 11,+ "execution_count": 12,"id": "e8adb325-2ad0-4a39-9bc2-d220ec3a29be","metadata": {},"outputs": [@@ -214,22 +249,22 @@"name": "stdout","output_type": "stream","text": [- "{'search_metadata': {'id': 'search_qVdXG2jzvrlqTzayeYoaOb8A',\n",+ "{'search_metadata': {'id': 'search_6Lpb2Z8vDqdsPRbrGkVgQzRy',\n"," 'status': 'Success',\n",- " 'created_at': '2023-09-25T15:22:30Z',\n",- " 'request_time_taken': 3.21,\n",- " 'parsing_time_taken': 0.03,\n",- " 'total_time_taken': 3.24,\n",+ " 'created_at': '2025-05-11T03:39:28Z',\n",+ " 'request_time_taken': 0.86,\n",+ " 'parsing_time_taken': 0.01,\n",+ " 'total_time_taken': 0.87,\n"," 'request_url': 'https://scholar.google.com/scholar?q=Large+Language+Models&hl=en',\n",- " 'html_url': 'https://www.searchapi.io/api/v1/searches/search_qVdXG2jzvrlqTzayeYoaOb8A.html',\n",- " 'json_url': 'https://www.searchapi.io/api/v1/searches/search_qVdXG2jzvrlqTzayeYoaOb8A'},\n",+ " 'html_url': 'https://www.searchapi.io/api/v1/searches/search_6Lpb2Z8vDqdsPRbrGkVgQzRy.html',\n",+ " 'json_url': 'https://www.searchapi.io/api/v1/searches/search_6Lpb2Z8vDqdsPRbrGkVgQzRy'},\n"," 'search_parameters': {'engine': 'google_scholar',\n"," 'q': 'Large Language Models',\n"," 'hl': 'en'},\n"," 'search_information': {'query_displayed': 'Large Language Models',\n",- " 'total_results': 6420000,\n",+ " 'total_results': 6390000,\n"," 'page': 1,\n",- " 'time_taken_displayed': 0.06},\n",+ " 'time_taken_displayed': 0.08},\n"," 'organic_results': [{'position': 1,\n"," 'title': 'ChatGPT for good? On opportunities and '\n"," 'challenges of large language models for '\n",@@ -245,15 +280,15 @@" 'we argue that large language models in '\n"," 'education require …',\n"," 'inline_links': {'cited_by': {'cites_id': '8166055256995715258',\n",- " 'total': 410,\n",- " 'link': 'https://scholar.google.com/scholar?cites=8166055256995715258&as_sdt=5,33&sciodt=0,33&hl=en'},\n",+ " 'total': 4675,\n",+ " 'link': 'https://scholar.google.com/scholar?cites=8166055256995715258&as_sdt=2005&sciodt=0,5&hl=en'},\n"," 'versions': {'cluster_id': '8166055256995715258',\n",- " 'total': 10,\n",- " 'link': 'https://scholar.google.com/scholar?cluster=8166055256995715258&hl=en&as_sdt=0,33'},\n",- " 'related_articles_link': 'https://scholar.google.com/scholar?q=related:uthwmf2nU3EJ:scholar.google.com/&scioq=Large+Language+Models&hl=en&as_sdt=0,33'},\n",- " 'resource': {'name': 'edarxiv.org',\n",+ " 'total': 16,\n",+ " 'link': 'https://scholar.google.com/scholar?cluster=8166055256995715258&hl=en&as_sdt=0,5'},\n",+ " 'related_articles_link': 'https://scholar.google.com/scholar?q=related:uthwmf2nU3EJ:scholar.google.com/&scioq=Large+Language+Models&hl=en&as_sdt=0,5'},\n",+ " 'resource': {'name': 'osf.io',\n"," 'format': 'PDF',\n",- " 'link': 'https://edarxiv.org/5er8f/download?format=pdf'},\n",+ " 'link': 'https://osf.io/preprints/edarxiv/5er8f/download'},\n"," 'authors': [{'name': 'E Kasneci',\n"," 'id': 'bZVkVvoAAAAJ',\n"," 'link': 'https://scholar.google.com/citations?user=bZVkVvoAAAAJ&hl=en&oi=sra'},\n",@@ -267,6 +302,82 @@" 'id': 'TjfQ8QkAAAAJ',\n"," 'link': 'https://scholar.google.com/citations?user=TjfQ8QkAAAAJ&hl=en&oi=sra'}]},\n"," {'position': 2,\n",+ " 'title': 'A survey on evaluation of large language '\n",+ " 'models',\n",+ " 'data_cid': 'o93zfHYlUTIJ',\n",+ " 'link': 'https://dl.acm.org/doi/abs/10.1145/3641289',\n",+ " 'publication': 'Y Chang, X Wang, J Wang, Y Wu, L Yang… - '\n",+ " 'ACM transactions on …, 2024 - dl.acm.org',\n",+ " 'snippet': '… 3.1 Natural Language Processing Tasks … '\n",+ " 'the development of language models, '\n",+ " 'particularly large language models, was to '\n",+ " 'enhance performance on natural language '\n",+ " 'processing tasks, …',\n",+ " 'inline_links': {'cited_by': {'cites_id': '3625720365842685347',\n",+ " 'total': 2864,\n",+ " 'link': 'https://scholar.google.com/scholar?cites=3625720365842685347&as_sdt=2005&sciodt=0,5&hl=en'},\n",+ " 'versions': {'cluster_id': '3625720365842685347',\n",+ " 'total': 8,\n",+ " 'link': 'https://scholar.google.com/scholar?cluster=3625720365842685347&hl=en&as_sdt=0,5'},\n",+ " 'related_articles_link': 'https://scholar.google.com/scholar?q=related:o93zfHYlUTIJ:scholar.google.com/&scioq=Large+Language+Models&hl=en&as_sdt=0,5'},\n",+ " 'resource': {'name': 'acm.org',\n",+ " 'format': 'PDF',\n",+ " 'link': 'https://dl.acm.org/doi/pdf/10.1145/3641289'},\n",+ " 'authors': [{'name': 'Y Chang',\n",+ " 'id': 'Hw-lrpAAAAAJ',\n",+ " 'link': 'https://scholar.google.com/citations?user=Hw-lrpAAAAAJ&hl=en&oi=sra'},\n",+ " {'name': 'X Wang',\n",+ " 'id': 'Q7Ieos8AAAAJ',\n",+ " 'link': 'https://scholar.google.com/citations?user=Q7Ieos8AAAAJ&hl=en&oi=sra'},\n",+ " {'name': 'J Wang',\n",+ " 'id': 'hBZ_tKsAAAAJ',\n",+ " 'link': 'https://scholar.google.com/citations?user=hBZ_tKsAAAAJ&hl=en&oi=sra'},\n",+ " {'name': 'Y Wu',\n",+ " 'id': 'KVeRu2QAAAAJ',\n",+ " 'link': 'https://scholar.google.com/citations?user=KVeRu2QAAAAJ&hl=en&oi=sra'},\n",+ " {'name': 'L Yang',\n",+ " 'id': 'go3sFxcAAAAJ',\n",+ " 'link': 'https://scholar.google.com/citations?user=go3sFxcAAAAJ&hl=en&oi=sra'}]},\n",+ " {'position': 3,\n",+ " 'title': 'A comprehensive overview of large language '\n",+ " 'models',\n",+ " 'data_cid': 'UDLkJGuOVl4J',\n",+ " 'link': 'https://arxiv.org/abs/2307.06435',\n",+ " 'publication': 'H Naveed, AU Khan, S Qiu, M Saqib, S '\n",+ " 'Anwar… - arXiv preprint arXiv …, 2023 - '\n",+ " 'arxiv.org',\n",+ " 'snippet': '… Large Language Models (LLMs) have recently '\n",+ " 'demonstrated remarkable capabilities in '\n",+ " 'natural language processing tasks and '\n",+ " 'beyond. This success of LLMs has led to a '\n",+ " 'large influx of …',\n",+ " 'inline_links': {'cited_by': {'cites_id': '6797777278393922128',\n",+ " 'total': 990,\n",+ " 'link': 'https://scholar.google.com/scholar?cites=6797777278393922128&as_sdt=2005&sciodt=0,5&hl=en'},\n",+ " 'versions': {'cluster_id': '6797777278393922128',\n",+ " 'total': 4,\n",+ " 'link': 'https://scholar.google.com/scholar?cluster=6797777278393922128&hl=en&as_sdt=0,5'},\n",+ " 'related_articles_link': 'https://scholar.google.com/scholar?q=related:UDLkJGuOVl4J:scholar.google.com/&scioq=Large+Language+Models&hl=en&as_sdt=0,5',\n",+ " 'cached_page_link': 'https://scholar.googleusercontent.com/scholar?q=cache:UDLkJGuOVl4J:scholar.google.com/+Large+Language+Models&hl=en&as_sdt=0,5'},\n",+ " 'resource': {'name': 'arxiv.org',\n",+ " 'format': 'PDF',\n",+ " 'link': 'https://arxiv.org/pdf/2307.06435'},\n",+ " 'authors': [{'name': 'H Naveed',\n",+ " 'id': 'k5dpooQAAAAJ',\n",+ " 'link': 'https://scholar.google.com/citations?user=k5dpooQAAAAJ&hl=en&oi=sra'},\n",+ " {'name': 'AU Khan',\n",+ " 'id': 'sbOhz2UAAAAJ',\n",+ " 'link': 'https://scholar.google.com/citations?user=sbOhz2UAAAAJ&hl=en&oi=sra'},\n",+ " {'name': 'S Qiu',\n",+ " 'id': 'OPNVthUAAAAJ',\n",+ " 'link': 'https://scholar.google.com/citations?user=OPNVthUAAAAJ&hl=en&oi=sra'},\n",+ " {'name': 'M Saqib',\n",+ " 'id': 'KvbLR3gAAAAJ',\n",+ " 'link': 'https://scholar.google.com/citations?user=KvbLR3gAAAAJ&hl=en&oi=sra'},\n",+ " {'name': 'S Anwar',\n",+ " 'id': 'vPJIHywAAAAJ',\n",+ " 'link': 'https://scholar.google.com/citations?user=vPJIHywAAAAJ&hl=en&oi=sra'}]},\n",+ " {'position': 4,\n"," 'title': 'Large language models in medicine',\n"," 'data_cid': 'Ph9AwHTmhzAJ',\n"," 'link': 'https://www.nature.com/articles/s41591-023-02448-8',\n",@@ -279,11 +390,15 @@" '(LLaMA) as its backend model 30 . Finally, '\n"," 'cheap imitations of …',\n"," 'inline_links': {'cited_by': {'cites_id': '3497017024792502078',\n",- " 'total': 25,\n",- " 'link': 'https://scholar.google.com/scholar?cites=3497017024792502078&as_sdt=5,33&sciodt=0,33&hl=en'},\n",+ " 'total': 2474,\n",+ " 'link': 'https://scholar.google.com/scholar?cites=3497017024792502078&as_sdt=2005&sciodt=0,5&hl=en'},\n"," 'versions': {'cluster_id': '3497017024792502078',\n",- " 'total': 3,\n",- " 'link': 'https://scholar.google.com/scholar?cluster=3497017024792502078&hl=en&as_sdt=0,33'}},\n",+ " 'total': 7,\n",+ " 'link': 'https://scholar.google.com/scholar?cluster=3497017024792502078&hl=en&as_sdt=0,5'},\n",+ " 'related_articles_link': 'https://scholar.google.com/scholar?q=related:Ph9AwHTmhzAJ:scholar.google.com/&scioq=Large+Language+Models&hl=en&as_sdt=0,5'},\n",+ " 'resource': {'name': 'google.com',\n",+ " 'format': 'PDF',\n",+ " 'link': 'https://drive.google.com/file/d/1FKEGsSZ9GYOeToeKpxB4m3atGRbC-TSm/view'},\n"," 'authors': [{'name': 'AJ Thirunavukarasu',\n"," 'id': '3qb1AYwAAAAJ',\n"," 'link': 'https://scholar.google.com/citations?user=3qb1AYwAAAAJ&hl=en&oi=sra'},\n",@@ -293,43 +408,132 @@" {'name': 'K Elangovan',\n"," 'id': 'BE_lVTQAAAAJ',\n"," 'link': 'https://scholar.google.com/citations?user=BE_lVTQAAAAJ&hl=en&oi=sra'}]},\n",- " {'position': 3,\n",- " 'title': 'Extracting training data from large language '\n",- " 'models',\n",- " 'data_cid': 'mEYsWK6bWKoJ',\n",- " 'link': 'https://www.usenix.org/conference/usenixsecurity21/presentation/carlini-extracting',\n",- " 'publication': 'N Carlini, F Tramer, E Wallace, M '\n",- " 'Jagielski… - 30th USENIX Security …, '\n",- " '2021 - usenix.org',\n",- " 'snippet': '… language model trained on scrapes of the '\n",- " 'public Internet, and are able to extract '\n",- " 'hundreds of verbatim text sequences from the '\n",- " 'model’… models are more vulnerable than '\n",- " 'smaller models. …',\n",- " 'inline_links': {'cited_by': {'cites_id': '12274731957504198296',\n",- " 'total': 742,\n",- " 'link': 'https://scholar.google.com/scholar?cites=12274731957504198296&as_sdt=5,33&sciodt=0,33&hl=en'},\n",- " 'versions': {'cluster_id': '12274731957504198296',\n",- " 'total': 8,\n",- " 'link': 'https://scholar.google.com/scholar?cluster=12274731957504198296&hl=en&as_sdt=0,33'},\n",- " 'related_articles_link': 'https://scholar.google.com/scholar?q=related:mEYsWK6bWKoJ:scholar.google.com/&scioq=Large+Language+Models&hl=en&as_sdt=0,33',\n",- " 'cached_page_link': 'https://scholar.googleusercontent.com/scholar?q=cache:mEYsWK6bWKoJ:scholar.google.com/+Large+Language+Models&hl=en&as_sdt=0,33'},\n",- " 'resource': {'name': 'usenix.org',\n",+ " {'position': 5,\n",+ " 'title': 'A watermark for large language models',\n",+ " 'data_cid': 'BlSyLHT4iiEJ',\n",+ " 'link': 'https://proceedings.mlr.press/v202/kirchenbauer23a.html',\n",+ " 'publication': 'J Kirchenbauer, J Geiping, Y Wen… - '\n",+ " 'International …, 2023 - '\n",+ " 'proceedings.mlr.press',\n",+ " 'snippet': '… We propose a watermarking framework for '\n",+ " 'proprietary language models. The … in the '\n",+ " 'language model just before it produces a '\n",+ " 'probability vector. The last layer of the '\n",+ " 'language model …',\n",+ " 'inline_links': {'cited_by': {'cites_id': '2417017327887471622',\n",+ " 'total': 774,\n",+ " 'link': 'https://scholar.google.com/scholar?cites=2417017327887471622&as_sdt=2005&sciodt=0,5&hl=en'},\n",+ " 'versions': {'cluster_id': '2417017327887471622',\n",+ " 'total': 13,\n",+ " 'link': 'https://scholar.google.com/scholar?cluster=2417017327887471622&hl=en&as_sdt=0,5'},\n",+ " 'related_articles_link': 'https://scholar.google.com/scholar?q=related:BlSyLHT4iiEJ:scholar.google.com/&scioq=Large+Language+Models&hl=en&as_sdt=0,5',\n",+ " 'cached_page_link': 'https://scholar.googleusercontent.com/scholar?q=cache:BlSyLHT4iiEJ:scholar.google.com/+Large+Language+Models&hl=en&as_sdt=0,5'},\n",+ " 'resource': {'name': 'mlr.press',\n"," 'format': 'PDF',\n",- " 'link': 'https://www.usenix.org/system/files/sec21-carlini-extracting.pdf'},\n",- " 'authors': [{'name': 'N Carlini',\n",- " 'id': 'q4qDvAoAAAAJ',\n",- " 'link': 'https://scholar.google.com/citations?user=q4qDvAoAAAAJ&hl=en&oi=sra'},\n",- " {'name': 'F Tramer',\n",- " 'id': 'ijH0-a8AAAAJ',\n",- " 'link': 'https://scholar.google.com/citations?user=ijH0-a8AAAAJ&hl=en&oi=sra'},\n",- " {'name': 'E Wallace',\n",- " 'id': 'SgST3LkAAAAJ',\n",- " 'link': 'https://scholar.google.com/citations?user=SgST3LkAAAAJ&hl=en&oi=sra'},\n",- " {'name': 'M Jagielski',\n",- " 'id': '_8rw_GMAAAAJ',\n",- " 'link': 'https://scholar.google.com/citations?user=_8rw_GMAAAAJ&hl=en&oi=sra'}]},\n",- " {'position': 4,\n",+ " 'link': 'https://proceedings.mlr.press/v202/kirchenbauer23a/kirchenbauer23a.pdf'},\n",+ " 'authors': [{'name': 'J Kirchenbauer',\n",+ " 'id': '48GJrbsAAAAJ',\n",+ " 'link': 'https://scholar.google.com/citations?user=48GJrbsAAAAJ&hl=en&oi=sra'},\n",+ " {'name': 'J Geiping',\n",+ " 'id': '206vNCEAAAAJ',\n",+ " 'link': 'https://scholar.google.com/citations?user=206vNCEAAAAJ&hl=en&oi=sra'},\n",+ " {'name': 'Y Wen',\n",+ " 'id': 'oUYfjg0AAAAJ',\n",+ " 'link': 'https://scholar.google.com/citations?user=oUYfjg0AAAAJ&hl=en&oi=sra'}]},\n",+ " {'position': 6,\n",+ " 'title': 'Welcome to the era of chatgpt et al. the '\n",+ " 'prospects of large language models',\n",+ " 'data_cid': '3UrgC1BmpV8J',\n",+ " 'link': 'https://link.springer.com/article/10.1007/s12599-023-00795-x',\n",+ " 'publication': 'T Teubner, CM Flath, C Weinhardt… - '\n",+ " 'Business & Information …, 2023 - '\n",+ " 'Springer',\n",+ " 'snippet': 'The emergence of Large Language Models '\n",+ " '(LLMs) in combination with easy-to-use '\n",+ " 'interfaces such as ChatGPT, Bing Chat, and '\n",+ " 'Google’s Bard represent both a Herculean '\n",+ " 'task and a …',\n",+ " 'inline_links': {'cited_by': {'cites_id': '6892027298743077597',\n",+ " 'total': 409,\n",+ " 'link': 'https://scholar.google.com/scholar?cites=6892027298743077597&as_sdt=2005&sciodt=0,5&hl=en'},\n",+ " 'versions': {'cluster_id': '6892027298743077597',\n",+ " 'total': 16,\n",+ " 'link': 'https://scholar.google.com/scholar?cluster=6892027298743077597&hl=en&as_sdt=0,5'},\n",+ " 'related_articles_link': 'https://scholar.google.com/scholar?q=related:3UrgC1BmpV8J:scholar.google.com/&scioq=Large+Language+Models&hl=en&as_sdt=0,5'},\n",+ " 'resource': {'name': 'springer.com',\n",+ " 'format': 'PDF',\n",+ " 'link': 'https://link.springer.com/content/pdf/10.1007/s12599-023-00795-x.pdf'},\n",+ " 'authors': [{'name': 'T Teubner',\n",+ " 'id': 'ZeCM1k8AAAAJ',\n",+ " 'link': 'https://scholar.google.com/citations?user=ZeCM1k8AAAAJ&hl=en&oi=sra'},\n",+ " {'name': 'CM Flath',\n",+ " 'id': '5Iy85HsAAAAJ',\n",+ " 'link': 'https://scholar.google.com/citations?user=5Iy85HsAAAAJ&hl=en&oi=sra'},\n",+ " {'name': 'C Weinhardt',\n",+ " 'id': 'lhfZxjAAAAAJ',\n",+ " 'link': 'https://scholar.google.com/citations?user=lhfZxjAAAAAJ&hl=en&oi=sra'}]},\n",+ " {'position': 7,\n",+ " 'title': 'Talking about large language models',\n",+ " 'data_cid': '3eYYI745r_0J',\n",+ " 'link': 'https://dl.acm.org/doi/abs/10.1145/3624724',\n",+ " 'publication': 'M Shanahan - Communications of the ACM, '\n",+ " '2024 - dl.acm.org',\n",+ " 'snippet': '… Recently, it has become commonplace to use '\n",+ " 'the term “large language model” both for the '\n",+ " 'generative models themselves and for the '\n",+ " 'systems in which they are embedded, '\n",+ " 'especially in …',\n",+ " 'inline_links': {'cited_by': {'cites_id': '18279892901315536605',\n",+ " 'total': 477,\n",+ " 'link': 'https://scholar.google.com/scholar?cites=18279892901315536605&as_sdt=2005&sciodt=0,5&hl=en'},\n",+ " 'versions': {'cluster_id': '18279892901315536605',\n",+ " 'total': 4,\n",+ " 'link': 'https://scholar.google.com/scholar?cluster=18279892901315536605&hl=en&as_sdt=0,5'},\n",+ " 'related_articles_link': 'https://scholar.google.com/scholar?q=related:3eYYI745r_0J:scholar.google.com/&scioq=Large+Language+Models&hl=en&as_sdt=0,5'},\n",+ " 'resource': {'name': 'acm.org',\n",+ " 'format': 'PDF',\n",+ " 'link': 'https://dl.acm.org/doi/pdf/10.1145/3624724'},\n",+ " 'authors': [{'name': 'M Shanahan',\n",+ " 'id': '00bnGpAAAAAJ',\n",+ " 'link': 'https://scholar.google.com/citations?user=00bnGpAAAAAJ&hl=en&oi=sra'}]},\n",+ " {'position': 8,\n",+ " 'title': 'Explainability for large language models: A '\n",+ " 'survey',\n",+ " 'data_cid': '0AqRKEINMw4J',\n",+ " 'link': 'https://dl.acm.org/doi/abs/10.1145/3639372',\n",+ " 'publication': 'H Zhao, H Chen, F Yang, N Liu, H Deng, H '\n",+ " 'Cai… - ACM Transactions on …, 2024 - '\n",+ " 'dl.acm.org',\n",+ " 'snippet': '… Let us consider a scenario where we have a '\n",+ " 'language model and we input a specific text '\n",+ " 'into the model. The model then produces a '\n",+ " 'classification output, such as sentiment …',\n",+ " 'inline_links': {'cited_by': {'cites_id': '1023176118142831312',\n",+ " 'total': 576,\n",+ " 'link': 'https://scholar.google.com/scholar?cites=1023176118142831312&as_sdt=2005&sciodt=0,5&hl=en'},\n",+ " 'versions': {'cluster_id': '1023176118142831312',\n",+ " 'total': 7,\n",+ " 'link': 'https://scholar.google.com/scholar?cluster=1023176118142831312&hl=en&as_sdt=0,5'},\n",+ " 'related_articles_link': 'https://scholar.google.com/scholar?q=related:0AqRKEINMw4J:scholar.google.com/&scioq=Large+Language+Models&hl=en&as_sdt=0,5'},\n",+ " 'resource': {'name': 'acm.org',\n",+ " 'format': 'PDF',\n",+ " 'link': 'https://dl.acm.org/doi/pdf/10.1145/3639372'},\n",+ " 'authors': [{'name': 'H Zhao',\n",+ " 'id': '9FobigIAAAAJ',\n",+ " 'link': 'https://scholar.google.com/citations?user=9FobigIAAAAJ&hl=en&oi=sra'},\n",+ " {'name': 'H Chen',\n",+ " 'id': 'DyYOgLwAAAAJ',\n",+ " 'link': 'https://scholar.google.com/citations?user=DyYOgLwAAAAJ&hl=en&oi=sra'},\n",+ " {'name': 'F Yang',\n",+ " 'id': 'RXFeW-8AAAAJ',\n",+ " 'link': 'https://scholar.google.com/citations?user=RXFeW-8AAAAJ&hl=en&oi=sra'},\n",+ " {'name': 'N Liu',\n",+ " 'id': 'Nir-EDYAAAAJ',\n",+ " 'link': 'https://scholar.google.com/citations?user=Nir-EDYAAAAJ&hl=en&oi=sra'},\n",+ " {'name': 'H Cai',\n",+ " 'id': 'Kz-r34UAAAAJ',\n",+ " 'link': 'https://scholar.google.com/citations?user=Kz-r34UAAAAJ&hl=en&oi=sra'}]},\n",+ " {'position': 9,\n"," 'title': 'Emergent abilities of large language models',\n"," 'data_cid': 'hG0iVOrOguoJ',\n"," 'link': 'https://arxiv.org/abs/2206.07682',\n",@@ -341,16 +545,16 @@" 'efficiency on a wide range of downstream '\n"," 'tasks. This paper instead discusses an …',\n"," 'inline_links': {'cited_by': {'cites_id': '16898296257676733828',\n",- " 'total': 621,\n",- " 'link': 'https://scholar.google.com/scholar?cites=16898296257676733828&as_sdt=5,33&sciodt=0,33&hl=en'},\n",+ " 'total': 3436,\n",+ " 'link': 'https://scholar.google.com/scholar?cites=16898296257676733828&as_sdt=2005&sciodt=0,5&hl=en'},\n"," 'versions': {'cluster_id': '16898296257676733828',\n",- " 'total': 12,\n",- " 'link': 'https://scholar.google.com/scholar?cluster=16898296257676733828&hl=en&as_sdt=0,33'},\n",- " 'related_articles_link': 'https://scholar.google.com/scholar?q=related:hG0iVOrOguoJ:scholar.google.com/&scioq=Large+Language+Models&hl=en&as_sdt=0,33',\n",- " 'cached_page_link': 'https://scholar.googleusercontent.com/scholar?q=cache:hG0iVOrOguoJ:scholar.google.com/+Large+Language+Models&hl=en&as_sdt=0,33'},\n",+ " 'total': 11,\n",+ " 'link': 'https://scholar.google.com/scholar?cluster=16898296257676733828&hl=en&as_sdt=0,5'},\n",+ " 'related_articles_link': 'https://scholar.google.com/scholar?q=related:hG0iVOrOguoJ:scholar.google.com/&scioq=Large+Language+Models&hl=en&as_sdt=0,5',\n",+ " 'cached_page_link': 'https://scholar.googleusercontent.com/scholar?q=cache:hG0iVOrOguoJ:scholar.google.com/+Large+Language+Models&hl=en&as_sdt=0,5'},\n"," 'resource': {'name': 'arxiv.org',\n"," 'format': 'PDF',\n",- " 'link': 'https://arxiv.org/pdf/2206.07682.pdf?trk=cndc-detail'},\n",+ " 'link': 'https://arxiv.org/pdf/2206.07682'},\n"," 'authors': [{'name': 'J Wei',\n"," 'id': 'wA5TK_0AAAAJ',\n"," 'link': 'https://scholar.google.com/citations?user=wA5TK_0AAAAJ&hl=en&oi=sra'},\n",@@ -362,232 +566,78 @@" 'link': 'https://scholar.google.com/citations?user=WMBXw1EAAAAJ&hl=en&oi=sra'},\n"," {'name': 'C Raffel',\n"," 'id': 'I66ZBYwAAAAJ',\n",- " 'link': 'https://scholar.google.com/citations?user=I66ZBYwAAAAJ&hl=en&oi=sra'},\n",- " {'name': 'B Zoph',\n",- " 'id': 'NL_7iTwAAAAJ',\n",- " 'link': 'https://scholar.google.com/citations?user=NL_7iTwAAAAJ&hl=en&oi=sra'}]},\n",- " {'position': 5,\n",- " 'title': 'A survey on evaluation of large language '\n",- " 'models',\n",- " 'data_cid': 'ZYohnzOz-XgJ',\n",- " 'link': 'https://arxiv.org/abs/2307.03109',\n",- " 'publication': 'Y Chang, X Wang, J Wang, Y Wu, K Zhu… - '\n",- " 'arXiv preprint arXiv …, 2023 - arxiv.org',\n",- " 'snippet': '… 3.1 Natural Language Processing Tasks … '\n",- " 'the development of language models, '\n",- " 'particularly large language models, was to '\n",- " 'enhance performance on natural language '\n",- " 'processing tasks, …',\n",- " 'inline_links': {'cited_by': {'cites_id': '8717195588046785125',\n",- " 'total': 31,\n",- " 'link': 'https://scholar.google.com/scholar?cites=8717195588046785125&as_sdt=5,33&sciodt=0,33&hl=en'},\n",- " 'versions': {'cluster_id': '8717195588046785125',\n",- " 'total': 3,\n",- " 'link': 'https://scholar.google.com/scholar?cluster=8717195588046785125&hl=en&as_sdt=0,33'},\n",- " 'cached_page_link': 'https://scholar.googleusercontent.com/scholar?q=cache:ZYohnzOz-XgJ:scholar.google.com/+Large+Language+Models&hl=en&as_sdt=0,33'},\n",- " 'resource': {'name': 'arxiv.org',\n",- " 'format': 'PDF',\n",- " 'link': 'https://arxiv.org/pdf/2307.03109'},\n",- " 'authors': [{'name': 'X Wang',\n",- " 'id': 'Q7Ieos8AAAAJ',\n",- " 'link': 'https://scholar.google.com/citations?user=Q7Ieos8AAAAJ&hl=en&oi=sra'},\n",- " {'name': 'J Wang',\n",- " 'id': 'YomxTXQAAAAJ',\n",- " 'link': 'https://scholar.google.com/citations?user=YomxTXQAAAAJ&hl=en&oi=sra'},\n",- " {'name': 'Y Wu',\n",- " 'id': 'KVeRu2QAAAAJ',\n",- " 'link': 'https://scholar.google.com/citations?user=KVeRu2QAAAAJ&hl=en&oi=sra'},\n",- " {'name': 'K Zhu',\n",- " 'id': 'g75dFLYAAAAJ',\n",- " 'link': 'https://scholar.google.com/citations?user=g75dFLYAAAAJ&hl=en&oi=sra'}]},\n",- " {'position': 6,\n",- " 'title': 'Evaluating large language models trained on '\n",- " 'code',\n",- " 'data_cid': '3tNvW3l5nU4J',\n",- " 'link': 'https://arxiv.org/abs/2107.03374',\n",- " 'publication': 'M Chen, J Tworek, H Jun, Q Yuan, HPO '\n",- " 'Pinto… - arXiv preprint arXiv …, 2021 - '\n",- " 'arxiv.org',\n",- " 'snippet': '… We introduce Codex, a GPT language model '\n",- " 'finetuned on publicly available code from '\n",- " 'GitHub, and study its Python code-writing '\n",- " 'capabilities. A distinct production version '\n",- " 'of Codex …',\n",- " 'inline_links': {'cited_by': {'cites_id': '5664817468434011102',\n",- " 'total': 941,\n",- " 'link': 'https://scholar.google.com/scholar?cites=5664817468434011102&as_sdt=5,33&sciodt=0,33&hl=en'},\n",- " 'versions': {'cluster_id': '5664817468434011102',\n",- " 'total': 2,\n",- " 'link': 'https://scholar.google.com/scholar?cluster=5664817468434011102&hl=en&as_sdt=0,33'},\n",- " 'related_articles_link': 'https://scholar.google.com/scholar?q=related:3tNvW3l5nU4J:scholar.google.com/&scioq=Large+Language+Models&hl=en&as_sdt=0,33',\n",- " 'cached_page_link': 'https://scholar.googleusercontent.com/scholar?q=cache:3tNvW3l5nU4J:scholar.google.com/+Large+Language+Models&hl=en&as_sdt=0,33'},\n",- " 'resource': {'name': 'arxiv.org',\n",- " 'format': 'PDF',\n",- " 'link': 'https://arxiv.org/pdf/2107.03374.pdf?trk=public_post_comment-text'},\n",- " 'authors': [{'name': 'M Chen',\n",- " 'id': '5fU-QMwAAAAJ',\n",- " 'link': 'https://scholar.google.com/citations?user=5fU-QMwAAAAJ&hl=en&oi=sra'},\n",- " {'name': 'J Tworek',\n",- " 'id': 'ZPuESCQAAAAJ',\n",- " 'link': 'https://scholar.google.com/citations?user=ZPuESCQAAAAJ&hl=en&oi=sra'},\n",- " {'name': 'Q Yuan',\n",- " 'id': 'B059m2EAAAAJ',\n",- " 'link': 'https://scholar.google.com/citations?user=B059m2EAAAAJ&hl=en&oi=sra'}]},\n",- " {'position': 7,\n",- " 'title': 'Large language models in machine translation',\n",- " 'data_cid': 'sY5m_Y3-0Y4J',\n",- " 'link': 'http://research.google/pubs/pub33278.pdf',\n",- " 'publication': 'T Brants, AC Popat, P Xu, FJ Och, J Dean '\n",- " '- 2007 - research.google',\n",- " 'snippet': '… the benefits of largescale statistical '\n",- " 'language modeling in ma… trillion tokens, '\n",- " 'resulting in language models having up to '\n",- " '300 … is inexpensive to train on large data '\n",- " 'sets and approaches the …',\n",- " 'type': 'PDF',\n",- " 'inline_links': {'cited_by': {'cites_id': '10291286509313494705',\n",- " 'total': 737,\n",- " 'link': 'https://scholar.google.com/scholar?cites=10291286509313494705&as_sdt=5,33&sciodt=0,33&hl=en'},\n",- " 'versions': {'cluster_id': '10291286509313494705',\n",- " 'total': 31,\n",- " 'link': 'https://scholar.google.com/scholar?cluster=10291286509313494705&hl=en&as_sdt=0,33'},\n",- " 'related_articles_link': 'https://scholar.google.com/scholar?q=related:sY5m_Y3-0Y4J:scholar.google.com/&scioq=Large+Language+Models&hl=en&as_sdt=0,33',\n",- " 'cached_page_link': 'https://scholar.googleusercontent.com/scholar?q=cache:sY5m_Y3-0Y4J:scholar.google.com/+Large+Language+Models&hl=en&as_sdt=0,33'},\n",- " 'resource': {'name': 'research.google',\n",- " 'format': 'PDF',\n",- " 'link': 'http://research.google/pubs/pub33278.pdf'},\n",- " 'authors': [{'name': 'FJ Och',\n",- " 'id': 'ITGdg6oAAAAJ',\n",- " 'link': 'https://scholar.google.com/citations?user=ITGdg6oAAAAJ&hl=en&oi=sra'},\n",- " {'name': 'J Dean',\n",- " 'id': 'NMS69lQAAAAJ',\n",- " 'link': 'https://scholar.google.com/citations?user=NMS69lQAAAAJ&hl=en&oi=sra'}]},\n",- " {'position': 8,\n",- " 'title': 'A watermark for large language models',\n",- " 'data_cid': 'BlSyLHT4iiEJ',\n",- " 'link': 'https://arxiv.org/abs/2301.10226',\n",- " 'publication': 'J Kirchenbauer, J Geiping, Y Wen, J '\n",- " 'Katz… - arXiv preprint arXiv …, 2023 - '\n",- " 'arxiv.org',\n",- " 'snippet': '… To derive this watermark, we examine what '\n",- " 'happens in the language model just before it '\n",- " 'produces a probability vector. The last '\n",- " 'layer of the language model outputs a vector '\n",- " 'of logits l(t). …',\n",- " 'inline_links': {'cited_by': {'cites_id': '2417017327887471622',\n",- " 'total': 104,\n",- " 'link': 'https://scholar.google.com/scholar?cites=2417017327887471622&as_sdt=5,33&sciodt=0,33&hl=en'},\n",- " 'versions': {'cluster_id': '2417017327887471622',\n",- " 'total': 4,\n",- " 'link': 'https://scholar.google.com/scholar?cluster=2417017327887471622&hl=en&as_sdt=0,33'},\n",- " 'related_articles_link': 'https://scholar.google.com/scholar?q=related:BlSyLHT4iiEJ:scholar.google.com/&scioq=Large+Language+Models&hl=en&as_sdt=0,33',\n",- " 'cached_page_link': 'https://scholar.googleusercontent.com/scholar?q=cache:BlSyLHT4iiEJ:scholar.google.com/+Large+Language+Models&hl=en&as_sdt=0,33'},\n",- " 'resource': {'name': 'arxiv.org',\n",- " 'format': 'PDF',\n",- " 'link': 'https://arxiv.org/pdf/2301.10226.pdf?curius=1419'},\n",- " 'authors': [{'name': 'J Kirchenbauer',\n",- " 'id': '48GJrbsAAAAJ',\n",- " 'link': 'https://scholar.google.com/citations?user=48GJrbsAAAAJ&hl=en&oi=sra'},\n",- " {'name': 'J Geiping',\n",- " 'id': '206vNCEAAAAJ',\n",- " 'link': 'https://scholar.google.com/citations?user=206vNCEAAAAJ&hl=en&oi=sra'},\n",- " {'name': 'Y Wen',\n",- " 'id': 'oUYfjg0AAAAJ',\n",- " 'link': 'https://scholar.google.com/citations?user=oUYfjg0AAAAJ&hl=en&oi=sra'},\n",- " {'name': 'J Katz',\n",- " 'id': 'yPw4WjoAAAAJ',\n",- " 'link': 'https://scholar.google.com/citations?user=yPw4WjoAAAAJ&hl=en&oi=sra'}]},\n",- " {'position': 9,\n",- " 'title': 'ChatGPT and other large language models are '\n",- " 'double-edged swords',\n",- " 'data_cid': 'So0q8TRvxhYJ',\n",- " 'link': 'https://pubs.rsna.org/doi/full/10.1148/radiol.230163',\n",- " 'publication': 'Y Shen, L Heacock, J Elias, KD Hentel, B '\n",- " 'Reig, G Shih… - Radiology, 2023 - '\n",- " 'pubs.rsna.org',\n",- " 'snippet': '… Large Language Models (LLMs) are deep '\n",- " 'learning models trained to understand and '\n",- " 'generate natural language. Recent studies '\n",- " 'demonstrated that LLMs achieve great success '\n",- " 'in a …',\n",- " 'inline_links': {'cited_by': {'cites_id': '1641121387398204746',\n",- " 'total': 231,\n",- " 'link': 'https://scholar.google.com/scholar?cites=1641121387398204746&as_sdt=5,33&sciodt=0,33&hl=en'},\n",- " 'versions': {'cluster_id': '1641121387398204746',\n",- " 'total': 3,\n",- " 'link': 'https://scholar.google.com/scholar?cluster=1641121387398204746&hl=en&as_sdt=0,33'},\n",- " 'related_articles_link': 'https://scholar.google.com/scholar?q=related:So0q8TRvxhYJ:scholar.google.com/&scioq=Large+Language+Models&hl=en&as_sdt=0,33'},\n",- " 'authors': [{'name': 'Y Shen',\n",- " 'id': 'XaeN2zgAAAAJ',\n",- " 'link': 'https://scholar.google.com/citations?user=XaeN2zgAAAAJ&hl=en&oi=sra'},\n",- " {'name': 'L Heacock',\n",- " 'id': 'tYYM5IkAAAAJ',\n",- " 'link': 'https://scholar.google.com/citations?user=tYYM5IkAAAAJ&hl=en&oi=sra'}]},\n",+ " 'link': 'https://scholar.google.com/citations?user=I66ZBYwAAAAJ&hl=en&oi=sra'}]},\n"," {'position': 10,\n",- " 'title': 'Pythia: A suite for analyzing large language '\n",- " 'models across training and scaling',\n",- " 'data_cid': 'aaIDvsMAD8QJ',\n",- " 'link': 'https://proceedings.mlr.press/v202/biderman23a.html',\n",- " 'publication': 'S Biderman, H Schoelkopf… - '\n",- " 'International …, 2023 - '\n",- " 'proceedings.mlr.press',\n",- " 'snippet': '… large language models, we prioritize '\n",- " 'consistency in model … out the most '\n",- " 'performance from each model. For example, we '\n",- " '… models, as it is becoming widely used for '\n",- " 'the largest models, …',\n",- " 'inline_links': {'cited_by': {'cites_id': '14127511396791067241',\n",- " 'total': 89,\n",- " 'link': 'https://scholar.google.com/scholar?cites=14127511396791067241&as_sdt=5,33&sciodt=0,33&hl=en'},\n",- " 'versions': {'cluster_id': '14127511396791067241',\n",- " 'total': 3,\n",- " 'link': 'https://scholar.google.com/scholar?cluster=14127511396791067241&hl=en&as_sdt=0,33'},\n",- " 'related_articles_link': 'https://scholar.google.com/scholar?q=related:aaIDvsMAD8QJ:scholar.google.com/&scioq=Large+Language+Models&hl=en&as_sdt=0,33',\n",- " 'cached_page_link': 'https://scholar.googleusercontent.com/scholar?q=cache:aaIDvsMAD8QJ:scholar.google.com/+Large+Language+Models&hl=en&as_sdt=0,33'},\n",- " 'resource': {'name': 'mlr.press',\n",+ " 'title': 'A systematic evaluation of large language '\n",+ " 'models of code',\n",+ " 'data_cid': '-iQSW0h72hYJ',\n",+ " 'link': 'https://dl.acm.org/doi/abs/10.1145/3520312.3534862',\n",+ " 'publication': 'FF Xu, U Alon, G Neubig, VJ Hellendoorn '\n",+ " '- Proceedings of the 6th ACM …, 2022 - '\n",+ " 'dl.acm.org',\n",+ " 'snippet': '… largest language models for code. We also '\n",+ " 'release PolyCoder, a large open-source '\n",+ " 'language model for code, trained exclusively '\n",+ " 'on code in 12 different programming '\n",+ " 'languages. In the …',\n",+ " 'inline_links': {'cited_by': {'cites_id': '1646764164453115130',\n",+ " 'total': 764,\n",+ " 'link': 'https://scholar.google.com/scholar?cites=1646764164453115130&as_sdt=2005&sciodt=0,5&hl=en'},\n",+ " 'versions': {'cluster_id': '1646764164453115130',\n",+ " 'total': 6,\n",+ " 'link': 'https://scholar.google.com/scholar?cluster=1646764164453115130&hl=en&as_sdt=0,5'},\n",+ " 'related_articles_link': 'https://scholar.google.com/scholar?q=related:-iQSW0h72hYJ:scholar.google.com/&scioq=Large+Language+Models&hl=en&as_sdt=0,5'},\n",+ " 'resource': {'name': 'acm.org',\n"," 'format': 'PDF',\n",- " 'link': 'https://proceedings.mlr.press/v202/biderman23a/biderman23a.pdf'},\n",- " 'authors': [{'name': 'S Biderman',\n",- " 'id': 'bO7H0DAAAAAJ',\n",- " 'link': 'https://scholar.google.com/citations?user=bO7H0DAAAAAJ&hl=en&oi=sra'},\n",- " {'name': 'H Schoelkopf',\n",- " 'id': 'XLahYIYAAAAJ',\n",- " 'link': 'https://scholar.google.com/citations?user=XLahYIYAAAAJ&hl=en&oi=sra'}]}],\n",- " 'related_searches': [{'query': 'large language models machine',\n",- " 'highlighted': ['machine'],\n",- " 'link': 'https://scholar.google.com/scholar?hl=en&as_sdt=0,33&qsp=1&q=large+language+models+machine&qst=ib'},\n",- " {'query': 'large language models pruning',\n",- " 'highlighted': ['pruning'],\n",- " 'link': 'https://scholar.google.com/scholar?hl=en&as_sdt=0,33&qsp=2&q=large+language+models+pruning&qst=ib'},\n",- " {'query': 'large language models multitask learners',\n",- " 'highlighted': ['multitask learners'],\n",- " 'link': 'https://scholar.google.com/scholar?hl=en&as_sdt=0,33&qsp=3&q=large+language+models+multitask+learners&qst=ib'},\n",- " {'query': 'large language models speech recognition',\n",- " 'highlighted': ['speech recognition'],\n",- " 'link': 'https://scholar.google.com/scholar?hl=en&as_sdt=0,33&qsp=4&q=large+language+models+speech+recognition&qst=ib'},\n",+ " 'link': 'https://dl.acm.org/doi/pdf/10.1145/3520312.3534862'},\n",+ " 'authors': [{'name': 'FF Xu',\n",+ " 'id': '1hXyfIkAAAAJ',\n",+ " 'link': 'https://scholar.google.com/citations?user=1hXyfIkAAAAJ&hl=en&oi=sra'},\n",+ " {'name': 'U Alon',\n",+ " 'id': 'QBn7vq8AAAAJ',\n",+ " 'link': 'https://scholar.google.com/citations?user=QBn7vq8AAAAJ&hl=en&oi=sra'},\n",+ " {'name': 'G Neubig',\n",+ " 'id': 'wlosgkoAAAAJ',\n",+ " 'link': 'https://scholar.google.com/citations?user=wlosgkoAAAAJ&hl=en&oi=sra'},\n",+ " {'name': 'VJ Hellendoorn',\n",+ " 'id': 'PfYrc5kAAAAJ',\n",+ " 'link': 'https://scholar.google.com/citations?user=PfYrc5kAAAAJ&hl=en&oi=sra'}]}],\n",+ " 'related_searches': [{'query': 'emergent large language models',\n",+ " 'highlighted': ['emergent'],\n",+ " 'link': 'https://scholar.google.com/scholar?hl=en&as_sdt=0,5&qsp=1&q=emergent+large+language+models&qst=ib'},\n",+ " {'query': 'large language models abilities',\n",+ " 'highlighted': ['abilities'],\n",+ " 'link': 'https://scholar.google.com/scholar?hl=en&as_sdt=0,5&qsp=2&q=large+language+models+abilities&qst=ib'},\n",+ " {'query': 'prompt large language models',\n",+ " 'highlighted': ['prompt'],\n",+ " 'link': 'https://scholar.google.com/scholar?hl=en&as_sdt=0,5&qsp=3&q=prompt+large+language+models&qst=ib'},\n",+ " {'query': 'large language models training '\n",+ " 'compute-optimal',\n",+ " 'highlighted': ['training compute-optimal'],\n",+ " 'link': 'https://scholar.google.com/scholar?hl=en&as_sdt=0,5&qsp=4&q=large+language+models+training+compute-optimal&qst=ib'},\n"," {'query': 'large language models machine translation',\n"," 'highlighted': ['machine translation'],\n",- " 'link': 'https://scholar.google.com/scholar?hl=en&as_sdt=0,33&qsp=5&q=large+language+models+machine+translation&qst=ib'},\n",- " {'query': 'emergent abilities of large language models',\n",- " 'highlighted': ['emergent abilities of'],\n",- " 'link': 'https://scholar.google.com/scholar?hl=en&as_sdt=0,33&qsp=6&q=emergent+abilities+of+large+language+models&qst=ir'},\n",- " {'query': 'language models privacy risks',\n",- " 'highlighted': ['privacy risks'],\n",- " 'link': 'https://scholar.google.com/scholar?hl=en&as_sdt=0,33&qsp=7&q=language+models+privacy+risks&qst=ir'},\n",- " {'query': 'language model fine tuning',\n",+ " 'link': 'https://scholar.google.com/scholar?hl=en&as_sdt=0,5&qsp=5&q=large+language+models+machine+translation&qst=ib'},\n",+ " {'query': 'large language models zero shot',\n",+ " 'highlighted': ['zero shot'],\n",+ " 'link': 'https://scholar.google.com/scholar?hl=en&as_sdt=0,5&qsp=6&q=large+language+models+zero+shot&qst=ib'},\n",+ " {'query': 'large language models chatgpt',\n",+ " 'highlighted': ['chatgpt'],\n",+ " 'link': 'https://scholar.google.com/scholar?hl=en&as_sdt=0,5&qsp=7&q=large+language+models+chatgpt&qst=ib'},\n",+ " {'query': 'fine tuning large language models',\n"," 'highlighted': ['fine tuning'],\n",- " 'link': 'https://scholar.google.com/scholar?hl=en&as_sdt=0,33&qsp=8&q=language+model+fine+tuning&qst=ir'}],\n",+ " 'link': 'https://scholar.google.com/scholar?hl=en&as_sdt=0,5&qsp=8&q=fine+tuning+large+language+models&qst=ib'}],\n"," 'pagination': {'current': 1,\n",- " 'next': 'https://scholar.google.com/scholar?start=10&q=Large+Language+Models&hl=en&as_sdt=0,33',\n",- " 'other_pages': {'2': 'https://scholar.google.com/scholar?start=10&q=Large+Language+Models&hl=en&as_sdt=0,33',\n",- " '3': 'https://scholar.google.com/scholar?start=20&q=Large+Language+Models&hl=en&as_sdt=0,33',\n",- " '4': 'https://scholar.google.com/scholar?start=30&q=Large+Language+Models&hl=en&as_sdt=0,33',\n",- " '5': 'https://scholar.google.com/scholar?start=40&q=Large+Language+Models&hl=en&as_sdt=0,33',\n",- " '6': 'https://scholar.google.com/scholar?start=50&q=Large+Language+Models&hl=en&as_sdt=0,33',\n",- " '7': 'https://scholar.google.com/scholar?start=60&q=Large+Language+Models&hl=en&as_sdt=0,33',\n",- " '8': 'https://scholar.google.com/scholar?start=70&q=Large+Language+Models&hl=en&as_sdt=0,33',\n",- " '9': 'https://scholar.google.com/scholar?start=80&q=Large+Language+Models&hl=en&as_sdt=0,33',\n",- " '10': 'https://scholar.google.com/scholar?start=90&q=Large+Language+Models&hl=en&as_sdt=0,33'}}}\n"+ " 'next': 'https://scholar.google.com/scholar?start=10&q=Large+Language+Models&hl=en&as_sdt=0,5',\n",+ " 'other_pages': {'2': 'https://scholar.google.com/scholar?start=10&q=Large+Language+Models&hl=en&as_sdt=0,5',\n",+ " '3': 'https://scholar.google.com/scholar?start=20&q=Large+Language+Models&hl=en&as_sdt=0,5',\n",+ " '4': 'https://scholar.google.com/scholar?start=30&q=Large+Language+Models&hl=en&as_sdt=0,5',\n",+ " '5': 'https://scholar.google.com/scholar?start=40&q=Large+Language+Models&hl=en&as_sdt=0,5',\n",+ " '6': 'https://scholar.google.com/scholar?start=50&q=Large+Language+Models&hl=en&as_sdt=0,5',\n",+ " '7': 'https://scholar.google.com/scholar?start=60&q=Large+Language+Models&hl=en&as_sdt=0,5',\n",+ " '8': 'https://scholar.google.com/scholar?start=70&q=Large+Language+Models&hl=en&as_sdt=0,5',\n",+ " '9': 'https://scholar.google.com/scholar?start=80&q=Large+Language+Models&hl=en&as_sdt=0,5',\n",+ " '10': 'https://scholar.google.com/scholar?start=90&q=Large+Language+Models&hl=en&as_sdt=0,5'}}}\n"]}],@@ -596,6 +646,14 @@"results = search.results(\"Large Language Models\")\n","pprint.pp(results)"]+ },+ {+ "cell_type": "code",+ "execution_count": null,+ "id": "11ab5938-e298-471d-96fc-50405ffad35c",+ "metadata": {},+ "outputs": [],+ "source": []}],"metadata": {@@ -614,7 +672,7 @@"name": "python","nbconvert_exporter": "python","pygments_lexer": "ipython3",- "version": "3.10.12"+ "version": "3.12.4"}},"nbformat": 4,