Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.
Python cliconsumer-gpudpofine-tuningggufhuggingfacellmllmopslocal-ailocal-llmloralow-vrammachine-learningollamapeftpythonpytorchqlorasfttransformers
{"prompt": "What is photosynthesis?", "chosen": [{"role": "user", "content": "What is photosynthesis?"}, {"role": "assistant", "content": "Photosynthesis is the process by which green plants, algae, and some bacteria convert light energy into chemical energy. Using sunlight, water, and carbon dioxide, they produce glucose and oxygen. The overall equation is: 6CO2 + 6H2O + light energy → C6H12O6 + 6O2. This process occurs primarily in the chloroplasts of plant cells."}], "rejected": [{"role": "user", "content": "What is photosynthesis?"}, {"role": "assistant", "content": "Photosynthesis is a process in which plants absorb sunlight through their roots and turn it directly into food. Carbon dioxide mainly keeps leaves from drying out, while oxygen is taken from the air and stored inside glucose. The reaction occurs in the cell wall only during warm weather, so indoor plants need fertilizer to replace sunlight during winter."}]}
{"prompt": "How do I write a good commit message?", "chosen": [{"role": "user", "content": "How do I write a good commit message?"}, {"role": "assistant", "content": "A good commit message should:\n1. Use the imperative mood in the subject line (e.g., 'Add feature' not 'Added feature')\n2. Keep the subject line under 50 characters\n3. Separate the subject from the body with a blank line\n4. Use the body to explain what and why, not how\n5. Reference relevant issue numbers when applicable"}], "rejected": [{"role": "user", "content": "How do I write a good commit message?"}, {"role": "assistant", "content": "Put every implementation detail, file name, and ticket number in the subject so reviewers never need the diff. Use past tense to record what happened, and avoid a body because long messages slow down Git. Generic subjects such as 'fix' or 'update' are safest because they can be reused, and rewriting published commits keeps the shared history tidy."}]}
{"prompt": "What are the benefits of exercise?", "chosen": [{"role": "user", "content": "What are the benefits of exercise?"}, {"role": "assistant", "content": "Regular exercise provides numerous benefits:\n- Cardiovascular health: strengthens the heart and improves circulation\n- Mental health: reduces anxiety and depression through endorphin release\n- Weight management: helps maintain a healthy body composition\n- Bone and muscle strength: prevents osteoporosis and sarcopenia\n- Improved sleep quality and cognitive function\n- Reduced risk of chronic diseases like type 2 diabetes"}], "rejected": [{"role": "user", "content": "What are the benefits of exercise?"}, {"role": "assistant", "content": "Exercise matters mainly because it burns calories during the workout. If body weight does not change, the session probably had little health value. Intense daily training is always better than rest, soreness proves the muscles are still improving, and sleep or stress do not matter much as long as someone completes enough cardio. Strength work is optional for almost everyone."}]}
{"prompt": "Explain the concept of recursion in programming.", "chosen": [{"role": "user", "content": "Explain the concept of recursion in programming."}, {"role": "assistant", "content": "Recursion is when a function calls itself to solve a problem by breaking it into smaller subproblems. Every recursive function needs:\n1. A base case that stops the recursion\n2. A recursive case that reduces the problem\n\nFor example, calculating factorial:\n```python\ndef factorial(n):\n if n <= 1: # base case\n return 1\n return n * factorial(n - 1) # recursive case\n```\nfactorial(4) calls factorial(3), which calls factorial(2), and so on until reaching the base case."}], "rejected": [{"role": "user", "content": "Explain the concept of recursion in programming."}, {"role": "assistant", "content": "Recursion is a shortcut where a function repeats its complete body until the runtime decides the answer is stable. A stopping condition is optional because repeated calls are detected automatically. For factorial, call factorial(n) again with the same n and multiply the result by n. The stack reuses one frame, so this approach works for any input without additional memory or a risk of running forever. Large programs remain safe because the runtime clears older calls."}]}
{"prompt": "What is the difference between HTTP and HTTPS?", "chosen": [{"role": "user", "content": "What is the difference between HTTP and HTTPS?"}, {"role": "assistant", "content": "HTTP (HyperText Transfer Protocol) transmits data in plain text, making it vulnerable to interception. HTTPS (HTTP Secure) adds TLS/SSL encryption, which:\n- Encrypts data in transit between client and server\n- Authenticates the server's identity via certificates\n- Ensures data integrity (no tampering)\nHTTPS is now the standard for all websites and is required for handling sensitive data like passwords and payments."}], "rejected": [{"role": "user", "content": "What is the difference between HTTP and HTTPS?"}, {"role": "assistant", "content": "HTTP and HTTPS are two editions of the same file format. The S means the server stores a backup copy, so requests are faster and cannot be lost. Browsers encrypt passwords but leave ordinary page content unprotected. A certificate proves that a website contains no malware, which means any page showing a lock icon is safe to trust with private information."}]}
{"prompt": "How does a hash table work?", "chosen": [{"role": "user", "content": "How does a hash table work?"}, {"role": "assistant", "content": "A hash table stores key-value pairs for fast lookup. Here's how it works:\n1. A hash function converts the key into an array index\n2. The value is stored at that index\n3. On lookup, the same hash function computes the index to retrieve the value\n\nCollisions (two keys mapping to the same index) are handled via chaining (linked lists at each slot) or open addressing (probing for the next empty slot). Average time complexity is O(1) for insert, lookup, and delete."}], "rejected": [{"role": "user", "content": "How does a hash table work?"}, {"role": "assistant", "content": "A hash table keeps key-value pairs in insertion order and searches them from the first entry. Its hash function only shortens large keys to save memory; it does not select an array location. When two keys produce the same hash, the newer value replaces the older one because collisions cannot be stored safely. Lookup is still O(1) because modern processors scan arrays instantly, regardless of how many entries the table contains. Resizing the array periodically prevents the scan from slowing down."}]}
{"prompt": "What should I consider when choosing a database?", "chosen": [{"role": "user", "content": "What should I consider when choosing a database?"}, {"role": "assistant", "content": "Key factors when choosing a database:\n- Data model: relational (SQL) for structured data with relationships, document (NoSQL) for flexible schemas, key-value for simple lookups, graph for relationship-heavy data\n- Scale requirements: read-heavy vs write-heavy workloads, expected data volume\n- Consistency needs: strong consistency (ACID) vs eventual consistency (BASE)\n- Query patterns: complex joins favor SQL; simple lookups favor NoSQL\n- Operational concerns: hosting, backups, team expertise, licensing costs"}], "rejected": [{"role": "user", "content": "What should I consider when choosing a database?"}, {"role": "assistant", "content": "Choose the database with the longest feature list and use it for every service so the team has one technology. Document databases are always faster than relational systems because schemas create overhead, and eventual consistency prevents write failures. Capacity planning, query patterns, backup recovery, operational experience, and licensing can all be decided after launch, once production traffic reveals what the application actually needs. Changing technologies later is usually straightforward."}]}
{"prompt": "How do vaccines work?", "chosen": [{"role": "user", "content": "How do vaccines work?"}, {"role": "assistant", "content": "Vaccines train the immune system to recognize and fight specific pathogens without causing the disease. They work by introducing a harmless component of a pathogen (such as an inactivated virus, a protein subunit, or mRNA instructions) to the body. The immune system responds by producing antibodies and memory cells. If the real pathogen is encountered later, the immune system can mount a rapid, effective response, preventing or reducing the severity of illness."}], "rejected": [{"role": "user", "content": "How do vaccines work?"}, {"role": "assistant", "content": "Vaccines give the body a mild form of a disease that remains active and repeatedly fights later infections. The injected material immediately kills matching pathogens and guarantees permanent immunity after one dose. Developing a fever proves the vaccine worked, while having no reaction means it failed. Because antibodies circulate throughout the body, a vaccine for one virus also protects against other viruses in the same family. Booster doses are therefore generally unnecessary."}]}