🐫 CAMEL: The first and the best multi-agent framework. Finding the Scaling Law of Agents. https://www.camel-ai.org
Python agentai-societiesartificial-intelligencecommunicative-aicooperative-aideep-learninglarge-language-modelsmulti-agent-systemsnatural-language-processing
feat: enhance source2synth (#1464)
Wendong-Fan4745e9950f8d
diff --git a/README.md b/README.mdindex 71004be9c..4aa46d548 100644--- a/README.md+++ b/README.md@@ -329,6 +329,8 @@ We implemented amazing research ideas from other works for you to build, compare- `Self-Instruct` from *Yizhong Wang et al.*: [SELF-INSTRUCT: Aligning Language Models with Self-Generated Instructions](https://arxiv.org/pdf/2212.10560). [[Example](https://github.com/camel-ai/camel/blob/master/examples/datagen/self_instruct/self_instruct.py)]+- `Source2Synth` from *Alisia Lupidi et al.*: [Source2Synth: Synthetic Data Generation and Curation Grounded in Real Data Sources](https://arxiv.org/abs/2409.08239). [[Example](https://github.com/camel-ai/camel/blob/master/examples/datagen/source2synth.py)]+## Other Research Works Based on Camel- [Agent Trust](http://agent-trust.camel-ai.org/): Can Large Language Model Agents Simulate Human Trust Behavior?diff --git a/camel/agents/multi_hop_generator_agent.py b/camel/agents/multi_hop_generator_agent.pyindex a232fce84..988342b9a 100644--- a/camel/agents/multi_hop_generator_agent.py+++ b/camel/agents/multi_hop_generator_agent.py@@ -22,17 +22,36 @@ from camel.agents.programmed_agent_instruction import (ProgrammedAgentInstructionResult,programmable_capability,)-from camel.messages import BaseMessage-from camel.synthetic_datagen.source2synth.models import (+from camel.datagen.source2synth.models import (ContextPrompt,MultiHopQA,)+from camel.messages import BaseMessageclass MultiHopGeneratorAgent(ProgrammableChatAgent):+ r"""An agent specialized in generating multi-hop question-answer pairs.++ This agent is designed to create complex questions that require multiple+ steps of reasoning to answer. It analyzes context to identify related+ facts and generates questions that require connecting these facts+ logically.++ Attributes:+ model_config (ConfigDict): Configuration for model behavior.+ system_message (BaseMessage): System message defining agent's role and+ instructions.+ """+model_config = ConfigDict(arbitrary_types_allowed=True)- def __init__(self, **kwargs: Any):+ def __init__(self, **kwargs: Any) -> None:+ r"""Initialize the MultiHopGeneratorAgent.++ Args:+ **kwargs (Any): Additional keyword arguments to pass to parent+ class.+ """super().__init__(**kwargs)system_text: str = textwrap.dedent(@@ -64,6 +83,19 @@ class MultiHopGeneratorAgent(ProgrammableChatAgent):def generate_multi_hop_qa(self, context: str) -> ProgrammedAgentInstructionResult[MultiHopQA]:+ r"""Generate a multi-hop question-answer pair from given context.++ Args:+ context (str): The input text context to generate QA from.++ Returns:+ ProgrammedAgentInstructionResult[MultiHopQA]: Result containing the+ generated question, reasoning steps, answer, and supporting+ facts.++ Raises:+ RuntimeError: If the agent fails to generate a response.+ """context_prompt = ContextPrompt(main_context=context, related_contexts=None)diff --git a/camel/agents/programmed_agent_instruction.py b/camel/agents/programmed_agent_instruction.pyindex 708d5997b..bf38d6710 100644--- a/camel/agents/programmed_agent_instruction.py+++ b/camel/agents/programmed_agent_instruction.py@@ -26,6 +26,16 @@ T = TypeVar('T')class ProgrammableAgentRequirement(Enum):+ r"""Requirements for programmable agent state.++ Defines the possible requirements that can be used to repair the state+ of a programmable agent.++ Attributes:+ LAST_MESSAGE_NOT_USER (str): Requires that the last message in the+ conversation was not from the user.+ """+LAST_MESSAGE_NOT_USER = "LAST_MESSAGE_NOT_USER"@@ -34,6 +44,11 @@ class ProgrammedAgentInstructionResult(BaseModel, Generic[T]):Contains the messages exchanged during execution and the computed value.The value type is specified by the generic type parameter T.++ Attributes:+ user_message (BaseMessage): The message sent by the user.+ agent_message (BaseMessage): The message sent by the agent.+ value (T): The computed result value of type T."""user_message: BaseMessage@@ -48,8 +63,7 @@ class AbstractProgrammableAgent(abc.ABC):A programmable agent is an agent that can be programmed to perform aspecific function or task. This class defines the interface for a- programmable- agent.+ programmable agent.These methods should be implemented in order to ensure the agent supportsthe necessary guarantees to enable a programming interface while@@ -68,16 +82,15 @@ class AbstractProgrammableAgent(abc.ABC):An atomic operation is an operation that is guaranteed tobe executed without interruption by any other operation.- If the operation fails or times out the agents state should be- unchanged.+ Args:+ callback (Callable[[], ProgrammedAgentInstructionResult[T]]): The+ operation to execute atomically.- If an operation is already in progress, this method should throw an- exception. (It is up to the caller to do any queuing)+ Returns:+ ProgrammedAgentInstructionResult[T]: The result of the operation.- If the agent is in a state where it can perform the operation,- it must leave the agent in a state where it can perform the- operation again. Though if state changes in successful operation- improve its ability to perform the operation, it should keep them.+ Raises:+ RuntimeError: If an operation is already in progress."""raise NotImplementedError@@ -86,10 +99,13 @@ class AbstractProgrammableAgent(abc.ABC):r"""Repair the state of the agent.Agents may have other non-atomic interfaces, such as a user interface,- or chat between other agents.+ or chat between other agents. This method should restore the agent to+ a state where it can perform operations according to the specified+ requirement.- This method should restore the agent to a state where it can perform- operations according to the specified requirement.+ Args:+ requirement (ProgrammableAgentRequirement): The requirement to+ repair the state for."""raise NotImplementedError@@ -99,10 +115,16 @@ def programmable_capability() -> Callable[..., ProgrammedAgentInstructionResult[T]]:r"""Decorator for programmable agent capabilities.- Wraps a method to ensure it is executed atomically via the agent's- run_atomic interface.- The decorated method must return a ProgrammedAgentInstructionResult with- appropriate type parameter.+ This decorator ensures that the decorated method is executed atomically+ and maintains the agent's state guarantees.++ Args:+ func (Callable[..., ProgrammedAgentInstructionResult[T]]): The method+ to decorate.++ Returns:+ Callable[..., ProgrammedAgentInstructionResult[T]]: The decorated+ method that ensures atomic execution."""@wraps(func)@@ -120,9 +142,20 @@ class ProgrammableChatAgent(ChatAgent, AbstractProgrammableAgent):Provides a default implementation of atomic execution using threading locksand basic state tracking for message roles. Implementing classes need toprovide specific repair logic for their use cases.++ Attributes:+ _operation_lock (threading.Lock): Lock for ensuring atomic operations.+ _last_message_role (Optional[str]): Role of the last message in the+ conversation."""- def __init__(self, **kwargs: Any):+ def __init__(self, **kwargs: Any) -> None:+ r"""Initialize the ProgrammableChatAgent.++ Args:+ **kwargs (Any): Additional keyword arguments to pass to parent+ class.+ """super().__init__(**kwargs)self._operation_lock = threading.Lock()self._last_message_role: Optional[str] = None@@ -130,6 +163,20 @@ class ProgrammableChatAgent(ChatAgent, AbstractProgrammableAgent):def run_atomic(self, callback: Callable[[], ProgrammedAgentInstructionResult[T]]) -> ProgrammedAgentInstructionResult[T]:+ r"""Run an atomic operation on the agent.++ Ensures thread-safe execution of the callback function by using a lock.++ Args:+ callback (Callable[[], ProgrammedAgentInstructionResult[T]]): The+ operation to execute atomically.++ Returns:+ ProgrammedAgentInstructionResult[T]: The result of the operation.++ Raises:+ RuntimeError: If an operation is already in progress.+ """if not self._operation_lock.acquire(blocking=False):raise RuntimeError("Operation already in progress")@@ -141,6 +188,14 @@ class ProgrammableChatAgent(ChatAgent, AbstractProgrammableAgent):self._operation_lock.release()def repair_state(self, requirement: ProgrammableAgentRequirement) -> None:+ r"""Repair the state of the agent.++ Implements basic state repair for message role requirements.++ Args:+ requirement (ProgrammableAgentRequirement): The requirement to+ repair the state for.+ """if requirement == ProgrammableAgentRequirement.LAST_MESSAGE_NOT_USER:if self._last_message_role == "user":raise NotImplementedError(diff --git a/camel/datagen/source2synth/__init__.py b/camel/datagen/source2synth/__init__.pynew file mode 100644index 000000000..e9ddca05f--- /dev/null+++ b/camel/datagen/source2synth/__init__.py@@ -0,0 +1,31 @@+# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========+# Licensed under the Apache License, Version 2.0 (the "License");+# you may not use this file except in compliance with the License.+# You may obtain a copy of the License at+#+# http://www.apache.org/licenses/LICENSE-2.0+#+# Unless required by applicable law or agreed to in writing, software+# distributed under the License is distributed on an "AS IS" BASIS,+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+# See the License for the specific language governing permissions and+# limitations under the License.+# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========+from .data_processor import (+ DataCurator,+ ExampleConstructor,+ UserDataProcessor,+)+from .models import MultiHopQA, ReasoningStep+from .user_data_processor_config import (+ ProcessorConfig,+)++__all__ = [+ "DataCurator",+ "ExampleConstructor",+ "ProcessorConfig",+ "UserDataProcessor",+ "ReasoningStep",+ "MultiHopQA",+]diff --git a/camel/synthetic_datagen/source2synth/data_processor.py b/camel/datagen/source2synth/data_processor.pysimilarity index 61%rename from camel/synthetic_datagen/source2synth/data_processor.pyrename to camel/datagen/source2synth/data_processor.pyindex 9780663c4..ec7d84ecc 100644--- a/camel/synthetic_datagen/source2synth/data_processor.py+++ b/camel/datagen/source2synth/data_processor.py@@ -15,33 +15,61 @@import randomfrom typing import Any, Dict, List, Optional, Sequence-import numpy as npfrom tqdm import tqdmfrom camel.agents.multi_hop_generator_agent import MultiHopGeneratorAgent-from camel.logger import get_logger-from camel.synthetic_datagen.source2synth.user_data_processor_config import (+from camel.datagen.source2synth.user_data_processor_config import (ProcessorConfig,)+from camel.logger import get_loggerlogger = get_logger(__name__)class UserDataProcessor:- r"""User Data Processor."""+ r"""A processor for generating multi-hop question-answer pairs from user+ data.++ This class handles the processing of text data to generate multi-hop+ question-answer pairs using either an AI model or rule-based approaches.+ It manages the entire pipeline from text preprocessing to dataset curation.++ Attributes:+ config (ProcessorConfig): Configuration for data processing parameters.+ rng (random.Random): Random number generator for reproducibility.+ multi_hop_agent (Optional[MultiHopGeneratorAgent]): Agent for+ generating QA pairs.+ """def __init__(self, config: Optional[ProcessorConfig] = None):+ r"""Initialize the UserDataProcessor.++ Args:+ config (Optional[ProcessorConfig], optional): Configuration for+ data processing. (default: :obj:`None`)+ """self.config = config or ProcessorConfig()- random.seed(self.config.seed)- np.random.seed(self.config.seed)+ self.rng = random.Random(self.config.seed)self.multi_hop_agent = (- MultiHopGeneratorAgent() if self.config.use_ai_model else None+ self.config.hop_generating_agent+ if self.config.use_ai_model+ else None)def process_text(self, text: str, source: str = "user_input") -> List[Dict[str, Any]]:- r"""Process a single text."""+ r"""Process a single text to generate multi-hop QA pairs.++ Args:+ text (str): The input text to process.+ source (str, optional): Source identifier for the text.+ (default: :obj:`"user_input"`)++ Returns:+ List[Dict[str, Any]]: List of processed examples with QA pairs and+ metadata.+ """# Convert text to standard formatraw_data = [{@@ -55,7 +83,7 @@ class UserDataProcessor:examples = constructor.construct_examples(raw_data)# Manage data- curator = DataCurator(self.config)+ curator = DataCurator(self.config, self.rng)final_dataset = curator.curate_dataset(examples)return final_dataset@@ -63,7 +91,20 @@ class UserDataProcessor:def process_batch(self, texts: List[str], sources: Optional[List[str]] = None) -> List[Dict[str, Any]]:- r"""Process multiple texts in batch."""+ r"""Process multiple texts in batch to generate multi-hop QA pairs.++ Args:+ texts (List[str]): List of input texts to process.+ sources (Optional[List[str]], optional): List of source+ identifiers. (default: :obj:`None`)++ Returns:+ List[Dict[str, Any]]: List of processed examples with QA pairs and+ metadata.++ Raises:+ ValueError: If length of sources doesn't match length of texts.+ """if sources is None:sources = ["user_input"] * len(texts)elif len(sources) != len(texts):@@ -82,27 +123,52 @@ class UserDataProcessor:examples = constructor.construct_examples(raw_data)# Manage data- curator = DataCurator(self.config)+ curator = DataCurator(self.config, self.rng)final_dataset = curator.curate_dataset(examples)return final_datasetclass ExampleConstructor:- r"""Example Constructor."""+ r"""Constructs training examples from raw text data.++ This class handles the construction of training examples by preprocessing+ text, extracting information pairs, and generating question-answer pairs.++ Attributes:+ config (ProcessorConfig): Configuration for example construction.+ multi_hop_agent (Optional[MultiHopGeneratorAgent]): Agent for QA+ generation.+ """def __init__(self,config: ProcessorConfig,multi_hop_agent: Optional[MultiHopGeneratorAgent] = None,):+ r"""Initialize the ExampleConstructor.++ Args:+ config (ProcessorConfig): Configuration for example construction.+ multi_hop_agent (Optional[MultiHopGeneratorAgent], optional):+ Agent for generating multi-hop QA pairs. (default: :obj:`None`)+ """self.config = configself.multi_hop_agent = multi_hop_agentdef construct_examples(self, raw_data: List[Dict[str, Any]]) -> List[Dict[str, Any]]:- r"""Construct training examples."""+ r"""Construct training examples from raw data.++ Args:+ raw_data (List[Dict[str, Any]]): List of raw data dictionaries+ containing text and metadata.++ Returns:+ List[Dict[str, Any]]: List of constructed examples with QA pairs+ and metadata.+ """logger.info("Starting to construct training examples...")examples = []@@ -135,7 +201,15 @@ class ExampleConstructor:return examplesdef _preprocess_text(self, text: str) -> str:- r"""Text preprocessing."""+ r"""Preprocess input text for example construction.++ Args:+ text (str): Input text to preprocess.++ Returns:+ str: Preprocessed text, or empty string if text fails quality+ checks.+ """if not isinstance(text, str):return ''@@ -156,7 +230,14 @@ class ExampleConstructor:return textdef _check_text_quality(self, text: str) -> bool:- r"""Check text quality."""+ r"""Check the quality of input text.++ Args:+ text (str): Text to check quality for.++ Returns:+ bool: True if text passes quality checks, False otherwise.+ """# 1. Basic quality checkif text.count('.') < 2: # Must have at least 2 sentencesreturn False@@ -171,7 +252,15 @@ class ExampleConstructor:return Truedef _extract_info_pairs(self, text: str) -> List[Dict[str, Sequence[str]]]:- r"""Extract information pairs and relationships."""+ r"""Extract information pairs and relationships from text.++ Args:+ text (str): Input text to extract information from.++ Returns:+ List[Dict[str, Sequence[str]]]: List of dictionaries containing+ premise, intermediate, conclusion, and related contexts.+ """# Split into sentencessentences = [s.strip() for s in text.split('.') if s.strip()]info_pairs = []@@ -200,7 +289,15 @@ class ExampleConstructor:def _generate_qa_pairs(self, info_pairs: List[Dict[str, Sequence[str]]]) -> List[Dict[str, str]]:- r"""Generate multi-hop question-answer pairs."""+ r"""Generate multi-hop question-answer pairs from information pairs.++ Args:+ info_pairs (List[Dict[str, Sequence[str]]]): List of information+ pairs extracted from text.++ Returns:+ List[Dict[str, str]]: List of generated QA pairs.+ """qa_pairs = []for pair in info_pairs:@@ -219,7 +316,15 @@ class ExampleConstructor:return qa_pairsdef _calculate_complexity(self, qa_pairs: List[Dict[str, Any]]) -> float:- r"""Calculate complexity of QA pairs."""+ r"""Calculate the complexity score for a set of QA pairs.++ Args:+ qa_pairs (List[Dict[str, Any]]): List of QA pairs to calculate+ complexity for.++ Returns:+ float: Complexity score between 0.0 and 1.0.+ """if not qa_pairs:return 0.0@@ -233,10 +338,10 @@ class ExampleConstructor:supporting_facts_count = len(qa.get('supporting_facts', []))# 3. Question length- question_length = len(qa['question'].split())+ question_length = len(qa.get('question', '').split())# 4. Answer length- answer_length = len(qa['answer'].split())+ answer_length = len(qa.get('answer', '').split())# Calculate complexity of a single QA pairqa_complexity = (@@ -256,15 +361,37 @@ class ExampleConstructor:class DataCurator:- r"""Data Manager."""+ r"""Manages and curates datasets of multi-hop question-answer pairs.++ This class handles dataset management tasks including quality filtering,+ complexity filtering, deduplication, and dataset sampling.- def __init__(self, config: ProcessorConfig):+ Attributes:+ config (ProcessorConfig): Configuration for data curation parameters.+ rng (random.Random): Random number generator for reproducible sampling.+ """++ def __init__(self, config: ProcessorConfig, rng: random.Random):+ r"""Initialize the DataCurator.++ Args:+ config (ProcessorConfig): Configuration for data curation.+ rng (random.Random): Random number generator for reproducibility.+ """self.config = config+ self.rng = rngdef curate_dataset(self, examples: List[Dict[str, Any]]) -> List[Dict[str, Any]]:- r"""Dataset management."""+ r"""Manage and curate a dataset through multiple filtering stages.++ Args:+ examples (List[Dict[str, Any]]): List of examples to curate.++ Returns:+ List[Dict[str, Any]]: Curated dataset meeting quality criteria.+ """logger.info("Starting dataset management...")# 1. Quality filtering@@ -296,7 +423,14 @@ class DataCurator:def _quality_filter(self, examples: List[Dict[str, Any]]) -> List[Dict[str, Any]]:- r"""Quality filtering."""+ r"""Filter examples based on quality criteria.++ Args:+ examples (List[Dict[str, Any]]): List of examples to filter.++ Returns:+ List[Dict[str, Any]]: Examples that pass quality checks.+ """filtered = []for example in examples:@@ -314,7 +448,14 @@ class DataCurator:return filtereddef _check_qa_quality(self, qa_pairs: List[Dict[str, str]]) -> bool:- r"""Check quality of QA pairs."""+ r"""Check the quality of question-answer pairs.++ Args:+ qa_pairs (List[Dict[str, str]]): List of QA pairs to check.++ Returns:+ bool: True if QA pairs meet quality criteria, False otherwise.+ """if not qa_pairs:return False@@ -335,7 +476,17 @@ class DataCurator:def _complexity_filter(self, examples: List[Dict[str, Any]]) -> List[Dict[str, Any]]:- r"""Complexity filtering."""+ """+ Filter examples based on complexity threshold.++ Removes examples with complexity scores below the configured threshold.++ Args:+ examples (List[Dict[str, Any]]): List of examples to filter.++ Returns:+ List[Dict[str, Any]]: Examples meeting complexity threshold.+ """return [examplefor example in examples@@ -346,7 +497,14 @@ class DataCurator:def _remove_duplicates(self, examples: List[Dict[str, Any]]) -> List[Dict[str, Any]]:- r"""Remove duplicates."""+ r"""Remove duplicate examples from the dataset.++ Args:+ examples (List[Dict[str, Any]]): List of examples to deduplicate.++ Returns:+ List[Dict[str, Any]]: Deduplicated examples.+ """seen = set()unique_examples = []@@ -366,8 +524,15 @@ class DataCurator:def _sample_dataset(self, examples: List[Dict[str, Any]]) -> List[Dict[str, Any]]:- r"""Sample to target dataset size."""+ r"""Sample examples to match target dataset size.++ Args:+ examples (List[Dict[str, Any]]): List of examples to sample from.++ Returns:+ List[Dict[str, Any]]: Sampled dataset of target size or smaller.+ """if len(examples) <= self.config.dataset_size:return examples- return random.sample(examples, self.config.dataset_size)+ return self.rng.sample(examples, self.config.dataset_size)diff --git a/camel/synthetic_datagen/source2synth/models.py b/camel/datagen/source2synth/models.pysimilarity index 73%rename from camel/synthetic_datagen/source2synth/models.pyrename to camel/datagen/source2synth/models.pyindex 568581411..b85b228f8 100644--- a/camel/synthetic_datagen/source2synth/models.py+++ b/camel/datagen/source2synth/models.py@@ -17,12 +17,30 @@ from pydantic import BaseModel, Fieldclass ReasoningStep(BaseModel):+ r"""A single step in a multi-hop reasoning process.++ Attributes:+ step (str): The textual description of the reasoning step.+ """+step: str = Field(..., description="A single step in the reasoning process.")class MultiHopQA(BaseModel):+ r"""A multi-hop question-answer pair with reasoning steps and supporting+ facts.++ Attributes:+ question (str): The question requiring multi-hop reasoning.+ reasoning_steps (List[ReasoningStep]): List of reasoning steps to+ answer.+ answer (str): The final answer to the question.+ supporting_facts (List[str]): List of facts supporting the reasoning.+ type (str): The type of question-answer pair.+ """+question: str = Field(..., description="The question that requires multi-hop reasoning.")@@ -57,6 +75,13 @@ class MultiHopQA(BaseModel):class ContextPrompt(BaseModel):+ r"""A context prompt for generating multi-hop question-answer pairs.++ Attributes:+ main_context (str): The primary context for generating QA pairs.+ related_contexts (Optional[List[str]]): Additional related contexts.+ """+main_context: str = Field(...,description="The main context for generating"diff --git a/camel/synthetic_datagen/source2synth/user_data_processor_config.py b/camel/datagen/source2synth/user_data_processor_config.pysimilarity index 85%rename from camel/synthetic_datagen/source2synth/user_data_processor_config.pyrename to camel/datagen/source2synth/user_data_processor_config.pyindex 9b99b9b83..8acc8cdaa 100644--- a/camel/synthetic_datagen/source2synth/user_data_processor_config.py+++ b/camel/datagen/source2synth/user_data_processor_config.py@@ -23,7 +23,15 @@ class ProcessorConfig(BaseModel):r"""Data processing configuration class"""def __repr__(self):- return "MultiHopGeneratorAgent()"+ return (+ f"ProcessorConfig("+ f"seed={self.seed}, min_length={self.min_length}, "+ f"max_length={self.max_length}, "+ f"complexity_threshold={self.complexity_threshold}, "+ f"dataset_size={self.dataset_size}, "+ f"use_ai_model={self.use_ai_model}"+ f")"+ )model_config = ConfigDict(validate_assignment=True,@@ -45,13 +53,6 @@ class ProcessorConfig(BaseModel):default=512, description="Maximum text length", gt=0)- quality_threshold: float = Field(- default=0.7,- description="Quality threshold for processing",- ge=0.0,- le=1.0,- )-complexity_threshold: float = Field(default=0.5,description="Complexity threshold for processing",diff --git a/examples/datagen/source2synth.py b/examples/datagen/source2synth.pyindex ba2549aed..65ee5ff53 100644--- a/examples/datagen/source2synth.py+++ b/examples/datagen/source2synth.py@@ -15,10 +15,10 @@import jsonimport logging-from camel.synthetic_datagen.source2synth.data_processor import (+from camel.datagen.source2synth.data_processor import (UserDataProcessor,)-from camel.synthetic_datagen.source2synth.user_data_processor_config import (+from camel.datagen.source2synth.user_data_processor_config import (ProcessorConfig,)@@ -43,7 +43,6 @@ def main():seed=42,min_length=50,max_length=1000,- quality_threshold=0.7,complexity_threshold=0.5,dataset_size=10,use_ai_model=True,@@ -171,3 +170,107 @@ def main():if __name__ == "__main__":main()+++'''+===============================================================================+Constructing examples: 100%|+███████████████████████████████████████████████████████████████████████████████+█| 1/1 [00:10<00:00, 10.98s/it]+Constructing examples: 100%|+███████████████████████████████████████████████████████████████████████████████+█| 3/3 [00:22<00:00, 7.64s/it]++=== Single Text Processing Example ===++Text 1:+Source: technology_evolution+Complexity: 0.88++Q&A Pairs:++Q&A Pair 1:+Type: multi_hop_qa+Question: How did the invention of transistors impact the development of+personal computers?+Reasoning Steps:+1. {'step': 'Identify the role of transistors in electronics.'}+2. {'step': 'Understand how transistors enabled the miniaturization of+computers.'}+3. {'step': 'Connect the miniaturization of computers to the creation of+personal computers in the 1980s.'}+4. {'step': 'Determine the overall impact of personal computers on work and+communication.'}+Answer: The invention of transistors allowed for smaller and more efficient+computers, which led to the development of personal computers in the 1980s,+transforming work and communication.+Supporting Facts:+1. Transistors are semiconductor devices that revolutionized electronics.+2. The miniaturization of computers was made possible by transistors.+3. Personal computers emerged in the 1980s as a result of smaller computer+designs.+4. Personal computers changed how people work and communicate.++Q&A Pair 2:+Type: multi_hop_qa+Question: What was the sequence of developments that led from transistors to+the internet?+Reasoning Steps:+1. {'step': 'Identify how transistors contributed to the development of+smaller and more efficient computers.'}+2. {'step': 'Explain how the miniaturization of computers resulted in the+creation of personal computers in the 1980s.'}+3. {'step': 'Discuss how personal computers transformed work and communication.+'}+4. {'step': 'Connect the transformation in communication to the rise of the+internet.'}+Answer: Transistors enabled smaller computers, which led to personal computers+in the 1980s, transforming communication and eventually giving rise to the+internet.+Supporting Facts:+1. Transistors are tiny semiconductor devices that made computers smaller and+more efficient.+2. The miniaturization of computers allowed for the creation of personal+computers in the 1980s.+3. Personal computers transformed how people work and communicate.+4. The digital revolution and personal computers contributed to the rise of+the internet, connecting billions worldwide.++Q&A Pair 3:+Type: multi_hop_qa+Question: How did the miniaturization of computers contribute to the+development of artificial intelligence systems today?+Reasoning Steps:+1. {'step': 'Identify the impact of miniaturization on the creation of+personal computers in the 1980s.'}+2. {'step': 'Explain how personal computers transformed communication and work.+'}+3. {'step': 'Connect the digital revolution and the rise of the internet to+the development of artificial intelligence.'}+4. {'step': 'Discuss how the interconnected network of the internet supports+AI systems in various industries.'}+Answer: The miniaturization of computers led to personal computers, which+transformed communication and work, and this digital revolution, along with+the internet, supports the development of artificial intelligence systems+today.+Supporting Facts:+1. Miniaturization of computers enabled the creation of personal computers in+the 1980s.+2. Personal computers transformed how people work and communicate.+3. The digital revolution led to the rise of the internet, connecting billions+of people.+4. The internet powers artificial intelligence systems that are reshaping+various industries.++=== Batch Processing Statistics ===+Total texts processed: 3+Total Q&A pairs generated: 9++=== Generation Statistics ===+AI-generated multi-hop Q&A count: 9+Template-generated multi-hop Q&A count: 0++Average reasoning steps: 4.00+Average complexity score: 0.90+===============================================================================+'''