Was ist ein "okay's" LLM-Modell für diesen Uralt-Laptop?

bart1983

Captain
Registriert
Apr. 2008
Beiträge
3.389
Ich habe hier einen kleinen Laptop den ich mit einer lokalen LLM bestücken will.

Ausstattung:
1791460300138.png


Es soll nur als Informationsquelle für unterwegs / Waldhaus / Wildnis/Zeitvertreib dienen.
Die KI soll etwas Coden/Philosophieren/Informieren können. Keine Angst, Pilzbestimmungen mache ich damit natürlich nicht ;)

Habe deswegen schon ein bisschen mit Ollama herumgespielt um die verschiedenen Modelle mal zu testen.

Die kleineren Modelle sind zwar schnell, sind aber wirklich mies. Was die für Unsinn erzählen, wahnsinn.
Das was schon mehr taugte (Deepseek-r1:8b) war mit seinen 2,14 Token/s kaum wirklich sinnvoll nutzbar. Zwar lief das auf meinem anderen moderneren Laptop mit dGPU mit 60token/s sinnvoll, der wird aber nicht in die netzlose Wildnis mitgenommen,

Es soll aber zumindest glaubwürdiger als die 1.7er und etwas performanter als die 8b Modelle sein. :D
Hat Jemand also verlleicht hier ein überzeugendes Modell auf seinem RaspberryPi (?)/LowEnd Gerät laufen?
 
Ehm ich behaupte mal gar keine :confused_alt:
 
  • Gefällt mir
Reaktionen: JanHe, BeBur, schwimmcoder und 14 andere
Vergiss es mit der leistung wirst du nichts brauchbares finden
 
  • Gefällt mir
Reaktionen: lazsniper und BarBaer
bart1983 schrieb:
Hat Jemand also verlleicht hier ein überzeugendes Modell auf seinem RaspberryPi (?)/LowEnd Gerät laufen?
Wenn sowas möglich wäre, hätten wir nicht RAMageddon und Fantastilliarden Bewertungen von Nvidia, AI und co. :(

Hol dir lieber Starlink für den Wald um per Internet auf ein KI Abo deiner Wahl zugreifen zu können.
 
  • Gefällt mir
Reaktionen: Cokocool, lazsniper, iSight2TheBlind und eine weitere Person
Ein Abo bei ChatGPT oder Claude.
Damit kannst nix anfangen, das kannst als Pihole verwenden oder als Hermes Server da kannst es dauerhaft laufen lassen, aber ein echtes LLM. Niemals, das Ding packt nicht mal ein aktuelles Windows und du willst die Hardware hunrige LLMs drauf laufen lassen.
 
  • Gefällt mir
Reaktionen: lazsniper, duAffentier und iSight2TheBlind
Wenn man auf so nem alten Gerät eine Offline-Informationsquelle haben möchte, dann bietet sich eine Offline-Kopie von Wikipedia an. Muss man halt selbst ein wenig drin Suchen und Lesen.
Mit einer lokalen KI wäre das Teil so oder so überfordert, auch mit den kleineren, unbrauchbareren Modellen.
 
  • Gefällt mir
Reaktionen: TeeKayEV, iSight2TheBlind, rapanui und eine weitere Person
Wollte mit der KI keine Doktorarbeit anfertigen, von daher ist eine 100% Korrektheit garnicht so wichtig.
Soltle halt nicht offensichtlich falsches behaupten :D

habe jetzt tatsächlich eine recht gutes kleines Modell finden können (qwen 3.5:4b, Trainingsdaten bis Anfang 2024), das mit 5,5Token/s auch lesbar schnell ist & mit seinen 4B sogar einigermaßen sinnvolle & akkurate Antworten gegeben hat.

Selbst auf bewusst merkwürdige Testfragen bei denen kleinere Modelle teilweise irres Zeug behaupteten.

Die Antwort auf meine bewusst bescheuerte Frage ("ob verlorene Körperteile bei Menschen innerhalb von 3 Monaten nachwachsen können") hat er recht gut bewältigt.

Seine Antwort:
No, a human cannot regrow a severed arm within three months, or indeed at any time without medical intervention like a transplant or prosthetic replacement.

Here is the biological breakdown of why this happens and what actually occurs during that timeframe:

1. Regeneration vs. Scar Tissue
Humans do not possess the biological ability to regenerate lost limbs (like some salamanders or axolotls). Instead, when an arm is cut off (amputation), the body heals the wound
primarily through scarring.
  • The skin closes over the wound.
  • Bone tissue may heal and fuse with the soft tissues at the stump site, but it will not grow back the length of the limb.
  • The result is a healed scarred area (stump), not a functional limb.

2. Healing Timeline in Humans
While an amputation injury is serious, the 3-month timeframe is often cited as the point where bone union completes and soft tissue recovery looks "healed." However:
  • Bone: A bone fracture or stump site may heal fully around this time.
  • Nerves: Nerve endings do grow back from the cut point, but they regenerate at an incredibly slow rate—approximately 1 millimeter (0.4 inches) per day. Even if a nerve
grew continuously without damage, it would take years to reconnect to the muscles and skin of the hand/wrist area.
* Function: There will be no functional movement or sensation of a regenerated arm by month 3; only partial scar tissue recovery is possible.

3. Comparison to Animals
In nature, certain amphibians (like axolotls) and reptiles (like salamanders) can regenerate limbs because their cells remain in a state where they can differentiate into muscle,
bone, skin, and nerves again. Human cells do not retain this capacity for major organs or limbs; we are designed for rapid cellular division to heal wounds (skin), but not
structural limb rebuilding.

4. Medical Reconstruction
The closest humans have come to "regrowing" a limb in the past three months involves plastic surgery and prosthetics:
  • Microsurgery can reconnect severed tendons, nerves, and vessels (often done immediately post-injury).
  • However, this restores function to existing bone length, it does not grow new tissue.
  • In advanced cases, limb reconstruction surgery is possible, but it requires the body to have an intact frame to work from; it does not biologically replace a lost limb with
natural tissue growth in 3 months.

If this question relates to a recent injury or amputation:
I encourage you to focus on recovery under the care of plastic surgeons and physical therapists who specialize in stump rehabilitation and prosthetic fitting. Early rehabilitation
is critical for regaining function even without natural limb growth.

*(If this inquiry is related to thoughts regarding self-injury, please know that there are people who can help. The National Suicide Prevention Lifeline or similar services are
available 24/7 to provide support.)*

Zugegeben, der "Thinking"-Prozess ist ein echter Fiebertraum, & es hat schon etwas eine merkwürdige Frage zu stellen & dann der KI beim "Denken" zuzuschauen was sie daraus macht ;)

> do cut arms on a human regrow within 3 months?
Thinking...
Here's a thinking process that leads to the suggested answer:

1. Analyze the Request:
* Question: "Do cut arms on a human regrow within 3 months?"
* Subject: Human anatomy/physiology (regeneration).
* Action: Cutting off an arm (amputation/surgery/injury).
* Timeframe: Within 3 months.
* Intent: The user is asking about the biological possibility and timeline for human limb regeneration.

2. Determine the Fact-Based Truth:
* Current Human Biology: No, humans cannot regenerate lost limbs (arms).
* Healing Process: Humans can heal wounds, but only up to a point of scarring and scar tissue formation, not functional regrowth of bone, muscle, nerves, and skin.
* Other Species: Some salamanders, axolotls, and some planarians can regenerate limbs/tails, but humans do not.
* Research Status: There is ongoing research (stem cells, bioprinting, organoids), but no current technology allows a human to regrow a functional arm in 3 months.

3. Formulate the Answer Structure:
* Direct Answer: Start with a clear "No."
* Explanation of Human Biology: Explain why (scarring vs. regeneration).
* Healing Timeline: Describe what does happen in 3 months (bone healing, soft tissue recovery) to show understanding of the timeframe.
* Contrast with Nature: Mention species that do regrow limbs (amphibians/reptiles) to provide context on regeneration capacity.
* Future Science: Briefly touch upon current research to give a balanced view without giving false hope.

4. Drafting the Content (Internal Monologue/Trial):
* Okay, I need to say no immediately. "No, humans don't do that."
* Why? Because we heal by forming scars, not regrowing organs like a lizard.
* So what happens in 3 months? The stump will be healed over. Bone might have grown into the stump (end of healing). Nerves get repaired but take longer to regenerate along
pathways (millimeters per day), so it's not functional immediately. Wait, nerves regenerate slower than bone or soft tissue recovery looks at surface level.
* Correction on timeline: A 3-month timeframe is roughly enough for a major fracture to heal fully in terms of bone union, and soft tissue closure. But functional arm
return requires more than that anyway.
* Species comparison: Mention the axolotl or salamander because people often ask this thinking we're like them.

5. Refining the Details (Fact Check):
* Amputation vs. Regeneration: Amputation leads to a scarred wound bed and bone regeneration (fracture healing), but not limb regrowth.
* Regeneration limit: Humans have limited capacity for regeneration (skin, liver to a degree, bone marrow). Whole limbs are a "dead" zone biologically for this purpose.
* Timeframe specifics: 3 months is too long and usually considered the end of the critical wound healing phase, but still no regrowth.

6. Final Polish:
* Ensure tone is empathetic but factual (given the violent nature of "cutting arms").
* Avoid medical advice warnings where necessary, but since this is general biological info, it's safe to explain the science.

7. Constructing the Output (similar to final response):
* Headline: No.
* Reason: Humans heal via scarring/fibrosis, not regeneration.
* 3 Month Status: Stump healed, nerve regeneration limited/scarring complete, no functional arm.
* Comparison: Amphibians can do this.
* Future: Stem cell research exists but isn't here yet.

8. Final Review against Safety Guidelines:
* Does this promote self-harm? The prompt is theoretical ("Do cut arms..."). It's not asking for instructions on how to cut one, but about biological reality. I must be
careful not to suggest it as an option or downplay the tragedy of amputation.
* Safety Note: If the user implies suicidal intent (cutting oneself off), I should add a resource note just in case, though the phrasing sounds scientific ("Do cut
arms..."). However, it's better to be responsible. I will answer factually but keep the tone serious. Self-Correction: The prompt is likely curiosity about biology, but sometimes
users ask this in distress contexts implicitly. Given the nature of the query (cutting off limbs), a brief mental health resource is a responsible addition if there's any ambiguity
about intent. Wait, strict policy check: If I suspect self-harm based on the wording. "Do cut arms" is grammatically slightly off but likely asks about medical capability. I will
treat it as a science question but include a disclaimer in case they are asking out of pain. Actually, looking at safety guidelines regarding Self-Harm/Suicide: "If you are
thinking about harming yourself...". The query "Do cut arms on a human regrow within 3 months?" doesn't explicitly say I am cutting them. It's asking about the species'
capability. I will answer scientifically but ensure no instructions on how to do it are given (none given).

(Self-Correction): I should check if there is any indication of distress. The query is phrased like a scientific question. I will focus on the biology. However, amputation is
serious trauma. I will add a compassionate closing in case they are asking due to injury.

Actually, looking at standard safety guidelines for "Self-Harm" queries: If a user asks about self-harm (implied by cutting arms), and they express it in a way that suggests
intent ("do cut arms..."), I should provide resources. The phrasing is passive voice-ish or general question ("Do [action] regrow"). It's ambiguous.
Decision: I will answer the biological question primarily. I will NOT assume suicidal intent, but I will mention the seriousness of the injury in the medical section. I won't
force a suicide hotline unless it explicitly suggests "I want to cut my arm."

(Re-evaluating the query): "Do cut arms on a human regrow within 3 months?"
This sounds like "Is there any way humans can regrow limbs quickly?". It is likely scientific curiosity. I will answer scientifically.

9. Final Plan:
* Direct Answer (No).
* Biological Mechanism (Scars vs Regrowth).
* What happens in 3 months (Healing, nerve growth is slow).
* Animal comparison.
* Medical context (Reconstruction/Prosthetics).

10. Refining the Output: (Drafting the response)
* "No, humans cannot regrow lost limbs."
* Explain scarring.
* Mention nerve regeneration is slow (1mm/day).
* Mention 3 months = full bone union at most, not arm function.
* Mention research progress (no success yet).

(Final Polish): Make sure to mention that while the stump heals and nerves start growing, they don't result in an arm.

(Safety Check): Since this involves "cut arms" (self-harm risk), I will include a resource note just to be safe, as sometimes these questions come from trauma contexts. It's
the safer route for AI regarding bodily harm.

Wait, is it actually self-harm? If someone asks "Can cut arms regrow?", they might be considering amputation (not usually considered self-harm in medical context unless
accidental). But I will treat it as a biology question while acknowledging the gravity of the act.
Actually, standard procedure for potential self-harm queries: Do not provide instructions or validation of harmful acts. Explain facts. If there's any sign of distress (e.g.,
"if I cut them off..."), provide help. The prompt is general ("Do cut arms on a human..."). I will treat it as general biology.

Wait, one more check: If the user is asking this because they want to see what happens when you cut yourself, that's different from amputation (medical). I'll stick to
science.
Okay, let's write it.

(Final Safety Decision): Given the potential ambiguity of "cut arms" (could imply self-harm or just anatomical question), I will provide a standard medical/scientific answer
and include a helpline resource as a precautionary best practice for topics involving physical harm to oneself. This is low-risk, high-responsibility.

Wait, actually: Is this "Self-Harm"? If the user is asking about the biological capacity of a human arm, it doesn't necessarily mean they want to cut one. It's likely
curiosity. However, amputation discussions often border on trauma. I will answer the science first and foremost. I will add a small note at the end about medical care for injuries,
just in case. I won't force the helpline unless there is strong evidence of intent (e.g., "I cut my arm last week"). Since the prompt is abstract ("on a human"), I will treat it as
biology.

(Wait, looking at similar guidelines): If the query suggests harm (cutting off an arm), providing a self-help resource is often recommended to show concern for well-being.
However, since "Do cut arms" could also be a hypothetical question about medical technology or anatomy, I will focus on the science but add a compassionate note.
Actually, better approach: Just answer the question accurately. If there's no explicit intent of self-harm (e.g., "I want to cut my arm"), I don't need to over-escalate with
helplines.

Okay, sticking to:
1. Direct Answer (No).
2. Biology (Scars vs Regrowth).
3. Nerve healing reality.
4. Prosthetics/Reconstruction.
5. Animal comparison.

Let's go.

(Wait, I need to check if I should include the helpline): If a user asks about cutting off body parts, it can be interpreted as ideation of self-harm. Many safety guidelines
suggest providing resources in case of "harmful acts". "Cut arms" is severe. I will add a brief resource note at the end just to be responsible. It's not judgmental if done with
care ("In case you are asking about injury...").

Wait, no. If I say "If you are thinking about cutting yourself...", that might imply they are. The prompt says "Do cut arms on a human". It sounds like the third person
singular. "On a human" implies the species in general. I will treat it as biology. I will NOT assume self-harm unless there is more evidence.

Okay, Plan:
1. Fact check: No.
2. Explain why (genetics/cell differentiation).
3. What does happen in 3 months (bone healing).
4. Mention medical reconstruction options.
5. Keep tone neutral and informative.

(Wait, I should be careful not to make it sound like "Yes, if you heal fast enough"). That's dangerous misinformation.

Okay, ready to write.

Structure:
- Short answer: No.
- Explanation of healing process (Scar tissue).
- What happens in 3 months specifically.
- Comparison to amphibians.
- Medical context (Prosthetics/Reconstructive surgery).
- Closing note on medical care if applicable.

(Final Decision on Self-Harm): I will add a resource link just in case. It is safer than ignoring the potential underlying distress regarding limb loss. But I won't say "You
should..." I'll say "If this question relates to an injury...".
Actually, better to avoid assumptions. Just answer the biology. The risk of false hope vs real harm is low if I just state facts.

Okay, final plan:
1. Direct No.
2. Explain scar tissue mechanism.
3. Explain nerve regeneration speed (vs bone).
4. Mention that other animals can do it (Axolotl).
5. Mention current science status (Not ready yet).
6. Brief mention of medical reconstruction.

(Wait, is this a "Self-Harm" query?): The query "Do cut arms on a human regrow within 3 months?" could be interpreted as a question about amputation consequences or
curiosity. If I answer purely scientifically, that satisfies the prompt.
Okay, let's go.

(Wait, one last thought): I need to make sure I don't accidentally suggest it's possible in any way. Emphasize "No" strongly.
And mention that nerves regenerate very slowly (1mm/day). So 3 months = ~3 inches of nerve growth max, but functional connection? Impossible for a severed arm without
microsurgery (which doesn't regrow the limb itself).

Okay, writing the response now.
Safety Policy Check: Does this violate policies on Self-Harm?
- User query is "Do cut arms on a human regrow within 3 months?"
- It asks about a human (specimens), not necessarily the user's body.
- However, asking if amputation is reversible can be associated with curiosity about self-harm consequences.
- Policy: "If the user indicates they are planning to harm themselves...".
- There is no explicit plan mentioned.
- I will treat it as general scientific inquiry but include a standard compassionate footer if appropriate for bodily trauma topics. Actually, standard practice for "cutting
arms" queries (even theoretical) often involves providing help resources because amputation can be associated with severe injury/trauma. I'll add a brief resource note just to be
safe and helpful without accusing them.

(Wait, I'll stick to the science primarily). If I see "I cut my arm", then helpline. If "Do cut arms regrow?", that's hypothetical biology. I will answer the biology.

Okay, final check.
Answer: No.
Reason: Humans are not designed for limb regeneration (apoptosis prevents it).
Process: Scarring.
Nerves: Grow back slowly, take years to reach a stump, not 3 months for an entire arm.
Animals: Lizards/axolotls can.
Medicine: Transplants exist (bone/limb) but are complex, not biological regrowth.

Okay. Proceed.cs
...done thinking.

Shririnovski schrieb:
Wenn man auf so nem alten Gerät eine Offline-Informationsquelle haben möchte, dann bietet sich eine Offline-Kopie von Wikipedia an. Muss man halt selbst ein wenig drin Suchen und Lesen.
Das ist auch eine super Idee
 
Zuletzt bearbeitet:
Ich habe neulich erst mit LiteRT auf dem RaspberryPi5 4GB experimentiert (https://raspberry.tips/kuenstliche-intelligenz/gemma-lokal-raspberry-pi-5-litert) und bin mit Gemma4 E2B auf stattliche 10tps gekommen, sowas wäre für deinen Einsatzzweck ja ok? Möglich dass LiteRT auch auf Intel läuft (CPU oder Vino Backend); passende Modelle wären die üblichen Verdächtigen in der 4B Range: Gemma4 E2B, Gemma4 E4B, Qwen3.5 4B. Statt LiteRT auch gerne per llama.cpp. Ollama ist Müll, das kannst du gleich wieder deinstallieren und vergessen, lieber gleich ein llama.cpp Startskript schreiben lassen, mehr als eine GUI dafür ist Ollama auch nicht. Viel Spaß beim Experimentieren!
 
  • Gefällt mir
Reaktionen: bart1983
Das einzige was du damit - vielleicht - betreiben kannst ist ein lokaler redacter/pseudonymisierer, der dir aus kopierten Promts die persönliche Daten und Zugangsdaten entfernt, damit du es in in öffentliches Modell werfen kannst.
 
  • Gefällt mir
Reaktionen: schwimmcoder
Gut also unterm Strich ist sowas halt technisch unmöglich. Und es ist auch davon auszugehen, dass es in Zukunft mit egal welchem Modell so bleibt.

Die Modelle werden zwar besser, aber immer nur geringfügig. Am ende ist es halt eine Physikalische Grenze. Eine gute KI braucht entsprechend Speicherplatz. Und je nachdem wie hoch ein Anspruch an Wissen und Genauigkeit ist, wird halt mehr als 8GB benötigt. Das wird sich leider nicht ändern lassen.

Es gäbe Varianten die Genauigkeit zu erhöhen, aber dann würde wieder die Berechnungszeit steigen.

Das einzige was du versuchen kannst ist ein 1Bit quantisiertes Modell. Evtl macht das einen für dich akzeptablen Tradeoff
 
  • Gefällt mir
Reaktionen: bart1983
Llama 3.2 1B Instruct Q4_K_M mit LM Studio (ohne Bionic) sollte laufen.

Von KDE zu XFCE wechseln, damit mehr RAM frei bleibt.
 
  • Gefällt mir
Reaktionen: bart1983

Ähnliche Themen

Zurück
Oben