{"id":2583,"date":"2026-06-15T16:57:49","date_gmt":"2026-06-15T14:57:49","guid":{"rendered":"https:\/\/wp.unil.ch\/iaunil\/local-ai-model-for-private-sensitive-and-confidential-data\/"},"modified":"2026-10-08T18:29:13","modified_gmt":"2026-10-08T16:29:13","slug":"local-ai-model-for-private-sensitive-and-confidential-data","status":"publish","type":"post","link":"https:\/\/wp.unil.ch\/iaunil\/en\/local-ai-model-for-private-sensitive-and-confidential-data\/","title":{"rendered":"Installing a local AI model for private and sensitive data (updated October 2026)"},"content":{"rendered":"\n

Why use local AI?<\/h3>\n\n

With LM Studio and a model downloaded to your computer<\/strong>, you can summarise, translate, rephrase a document or ask questions about its contents, offline. The application and these open-weight models are free, with no account required. Internet access is needed for installation, downloads and updates.<\/p>\n\n

At UNIL, sensitive data as defined under Swiss legislation, in particular the Vaud data protection act (LPrD) (health, opinions, etc.), must be processed with a local solution that is authorised and appropriate for that data. The AI_GO<\/a> questionnaire helps you choose an authorised tool, alongside the UNIL AI FAQ (in French)<\/a>; you must also follow the rules of your course, faculty or publisher. For information covered by official secrecy that does not include sensitive data, UNIL\u2019s institutional Copilot<\/a> with your UNIL account and the UNIL [S\u00e9curis\u00e9] agent may be suitable, except for HR data or delicate personal data. For sensitive research data, first contact the DCSR, the research computing support unit, via helpdesk@unil.ch.<\/p>\n\n

Let us be honest.<\/strong> These small models can make mistakes and remain less capable than the best online models. Their main benefit is confidentiality, with no guarantee of energy savings.<\/p>\n\n

Install LM Studio<\/h3>\n\n

Ideally, plan for 16 GB of RAM<\/strong>, about 6 to 8 GB of disk space per compressed model<\/strong>, plus the application. On Mac: an Apple Silicon chip (M1 or newer) and at least macOS 14; Intel Macs are not supported. On Windows: an AVX2-compatible Intel\/AMD processor or Snapdragon X Elite (official requirements<\/a>). A dedicated graphics card can speed up responses, but is not essential.<\/p>\n\n

    \n
  1. On lmstudio.ai\/download<\/a>, scroll down to the LM Studio<\/strong> section, below Bionic, and choose \u201cDownload LM Studio\u201d<\/strong>.<\/li>\n
  2. Install then open the application. Labels vary depending on the language and version: look for the function described.<\/li>\n<\/ol>\n\n

    Which model should you choose?<\/h3>\n\n

    When in doubt, start with Qwen3.5 9B<\/strong>, the choice we made in our tests. On a modest computer, a long document can take more than ten minutes: this wait does not necessarily mean something has gone wrong.<\/p>\n\n

    \n\n\n\n\n\n\n
    Model<\/th>\nWhen should you choose it?<\/th>\n<\/tr>\n<\/thead>\n
    Qwen3.5 9B<\/strong><\/td>\nA good balance between quality and memory use to get started with documents; responses can be slow.<\/td>\n<\/tr>\n
    Gemma 4 E4B<\/strong><\/td>\nTo prioritise speed, with more limited capabilities and no guaranteed memory savings.<\/td>\n<\/tr>\n
    Gemma 4 12B<\/strong><\/td>\nTo prioritise overall quality: the highest score of the three in the Artificial Analysis<\/a> index checked on 8 October 2026 (estimate). More memory-intensive, especially for long documents.<\/td>\n<\/tr>\n<\/tbody>\n<\/table><\/figure>\n\n

    File size is not the whole picture: the document and conversation also use memory. Check the memory estimate when loading and any warnings from LM Studio. You can keep several models installed if disk space allows. Our tests were run on a MacBook Air M1 with 16 GB of RAM, using LM Studio 0.4.16.<\/p>\n\n

    Download the model<\/h3>\n\n
      \n
    1. Open the \u201cDiscover\u201d<\/strong> magnifying glass and search for the full name of your chosen model, for example qwen3.5 9b<\/strong>. Select the result with exactly that name.<\/li>\n
    2. Download the default version if it is shown as compatible with your computer. For Qwen, if you need to choose: MLX 4bit<\/strong> on Apple Silicon Macs or GGUF Q4_K_M<\/strong> on Windows. For another model, choose a compatible 4-bit<\/strong> version. These compressed versions take up less space, sometimes at the cost of quality.<\/li>\n
    3. After downloading, click \u201cUse in New Chat\u201d<\/strong> to load the model and open a conversation.<\/li>\n<\/ol>\n\n
      \"Searching
      The screenshots show Gemma 4 E4B; for the recommended choice, select Qwen3.5 9B.<\/figcaption><\/figure>\n\n

      Set the context for a document<\/h3>\n\n

      The context<\/strong> is the amount of text the model can keep in view, including the document and conversation. It is measured in tokens<\/em>, pieces of words. If the document is too long, LM Studio only passes on excerpts, so a summary may omit sections.<\/p>\n\n

        \n
      1. Click the cog to the left of the model name at the top<\/strong>. If needed, enable Settings \u2192 Developer<\/strong> to show the advanced settings.<\/li>\n
      2. Try 32,000<\/strong> in \u201cContext Length\u201d<\/strong>, if memory allows, then \u201cReload to apply changes\u201d<\/strong>. With Gemma 4 12B on 16 GB, start closer to 16,000<\/strong>.<\/li>\n
      3. If memory is insufficient, close demanding applications or lower this value and work in sections.<\/li>\n<\/ol>\n\n
        \"Context
        The context setting and the button to reload the model.<\/figcaption><\/figure>\n\n

        If the model says \u201cI cannot see any document\u201d<\/strong>, check that the file has loaded and try a larger context if memory allows. After this change, open a new conversation, attach the document and send the request again. 32,000 does not guarantee a complete summary.<\/strong> For each request, check the processing details: \u201cinject-full-content\u201d means all extracted text is sent; \u201cretrieval\u201d means only a few excerpts are sent. For an overall summary, if \u201cretrieval\u201d appears, if these details are not visible or if sections are missing, copy and paste the text in sections, each in a new conversation. Ask \u201cSummarise this section\u201d, then combine the summaries and check that all sections are covered.<\/p>\n\n

        Work on your document<\/h3>\n\n

        For sensitive data, use an up-to-date computer with both the disk and backups encrypted, and lock your session when you are away. Choose a storage location authorised for this data<\/strong>, outside OneDrive, iCloud or an unauthorised cloud: Desktop and Documents are often synced. Use the downloaded local model, without web features or remote services; you can disconnect from the Internet after the downloads.<\/p>\n\n

        PDFs: watch out for ignored content.<\/strong> In this workflow, only extracted text is passed on, not images or charts. Scanned pages without selectable text may be ignored, leading to an incomplete or incorrect response. Prefer the original text file; do not send a sensitive document to an online converter.<\/p>\n\n

        Before sending<\/strong>, with Qwen and a long document, try turning off \u201cThink\u201d<\/strong>, below the message box. This may speed up the response, at the cost of less thorough reasoning. If reasoning continues or the wait becomes too long, work in sections.<\/p>\n\n

          \n
        1. Open a new conversation for each document<\/strong> and drag the .pdf, .docx or .txt<\/strong> file into it. Wait until it has finished loading.<\/li>\n
        2. For example, type: \u201cSummarise this report in 5 points.\u201d<\/strong><\/li>\n
        3. Check the response against the original document: figures, names and sections covered. Mention the use of AI if its contribution is significant.<\/li>\n<\/ol>\n\n
          \"A
          Example with Gemma: the attached PDF, the request and the response.<\/figcaption><\/figure>\n\n

          After use.<\/strong> Delete conversations you no longer need. Copies or excerpts may remain in LM Studio: ask Central IT Services (helpdesk@unil.ch) for help removing them. The summary retains the sensitivity of the source document: protect it in the same way.<\/p>\n\n

          Alternatives<\/h3>\n\n

          Ollama also lets you use local models. For an institutional solution, or if your computer is not powerful enough, contact Central IT Services (helpdesk@unil.ch).<\/p>\n","protected":false},"excerpt":{"rendered":"

          Local AIs are increasingly powerful and attractive, all while keeping your sensitive data on your own computer rather than on remote servers.<\/p>\n","protected":false},"author":1002618,"featured_media":2211,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_seopress_titles_title":"","_seopress_titles_desc":"","_seopress_robots_index":"","_seopress_robots_follow":"","_seopress_robots_imageindex":"","_seopress_robots_snippet":"","_seopress_robots_primary_cat":"","_seopress_robots_breadcrumbs":"","_seopress_robots_freeze_modified_date":"","_seopress_robots_custom_modified_date":"","_seopress_robots_canonical":"","_seopress_social_fb_title":"","_seopress_social_fb_desc":"","_seopress_social_fb_img":"","_seopress_social_fb_img_attachment_id":0,"_seopress_social_fb_img_width":0,"_seopress_social_fb_img_height":0,"_seopress_social_twitter_title":"","_seopress_social_twitter_desc":"","_seopress_social_twitter_img":"","_seopress_social_twitter_img_attachment_id":0,"_seopress_social_twitter_img_width":0,"_seopress_social_twitter_img_height":0,"_seopress_redirections_value":"","_seopress_redirections_enabled":"","_seopress_redirections_enabled_regex":"","_seopress_redirections_logged_status":"","_seopress_redirections_param":"","_seopress_redirections_type":0,"_seopress_analysis_target_kw":"","_seopress_news_disabled":"","_seopress_video_disabled":"","_seopress_video":[],"_seopress_pro_schemas_manual":[],"_seopress_pro_rich_snippets_disable_all":"","_seopress_pro_rich_snippets_disable":[],"_seopress_pro_schemas":[],"footnotes":""},"categories":[22],"tags":[],"class_list":["post-2583","post","type-post","status-publish","format-standard","has-post-thumbnail","category-resources"],"_links":{"self":[{"href":"https:\/\/wp.unil.ch\/iaunil\/en\/wp-json\/wp\/v2\/posts\/2583","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wp.unil.ch\/iaunil\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wp.unil.ch\/iaunil\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/wp.unil.ch\/iaunil\/en\/wp-json\/wp\/v2\/users\/1002618"}],"replies":[{"embeddable":true,"href":"https:\/\/wp.unil.ch\/iaunil\/en\/wp-json\/wp\/v2\/comments?post=2583"}],"version-history":[{"count":5,"href":"https:\/\/wp.unil.ch\/iaunil\/en\/wp-json\/wp\/v2\/posts\/2583\/revisions"}],"predecessor-version":[{"id":4432,"href":"https:\/\/wp.unil.ch\/iaunil\/en\/wp-json\/wp\/v2\/posts\/2583\/revisions\/4432"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/wp.unil.ch\/iaunil\/en\/wp-json\/wp\/v2\/media\/2211"}],"wp:attachment":[{"href":"https:\/\/wp.unil.ch\/iaunil\/en\/wp-json\/wp\/v2\/media?parent=2583"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wp.unil.ch\/iaunil\/en\/wp-json\/wp\/v2\/categories?post=2583"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wp.unil.ch\/iaunil\/en\/wp-json\/wp\/v2\/tags?post=2583"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}