문의하기 한국어|English
인사이트로 돌아가기
Tech & Insights

Understanding Private AI: Using AI Without Handing Over Your Data

PR TeamPR Team · DESILO
·

Tech & Insight — The Homomorphic Encryption & Private AI Series (4 parts, complete)

At a glance (TL;DR)

  • Private AI refers to the whole family of approaches for using AI's capabilities while control of the data stays entirely with its owner. The heart of it is the guarantee, not the deployment model. Prompts and documents must not be exposed — not even to the model operator.
  • Private AI is rising now for three reasons: agentic AI, whose structure turns an organization's internal data wholesale into model input; tightening regulation; and stalled AI adoption in regulated industries.
  • The answers ultimately divide over one question: whom do you trust? Private deployment trusts your own infrastructure; TEEs trust the chip maker; MPC trusts that servers will not collude. Fully homomorphic encryption (FHE) erases the question itself. The entire computation runs on ciphertext, so there is no party that could see the data in the first place.
  • Speed, homomorphic encryption's old weakness, is closing fast. The evidence: THOR, which brought encrypted LLM inference down to minutes, and the fifth-generation GL Scheme, which redesigned the scheme itself for matrix operations. DESILO is building these technologies into the DESILO Private AI product family.

Table of contents

  1. What is Private AI: a property, not a promise
  2. Why now: the last bottleneck in AI adoption
  3. What leaks: the exposure points in the AI pipeline
  4. The first answer, bringing the model to the data: private deployment
  5. The second answer, moving the trust: TEE, MPC, federated learning, and differential privacy
  6. The third answer, removing the trust: homomorphic encryption
  7. Proof of practicality: what THOR and GL changed
  8. What you can build: from Secure RAG to agentic AI
  9. Frequently asked questions (FAQ)
  10. References

1. What is Private AI: a property, not a promise

In most AI services today, privacy is a promise. "We won't use your inputs for training," "we delete data after a set period" — you have to rely on terms and contracts, and on the operator's good faith in keeping them. Promises are useful, but they are promises that can break in the face of misconfiguration, insiders, breaches, and policy changes.

Private AI is the attempt to turn privacy from a promise into a property of the system. It makes the question of who can see the data something decided by the architecture itself, not by operating policy. By this standard, Private AI is not the name of any particular deployment model. An AI installed on the corporate network can be one form of Private AI, but the location of installation is not itself a guarantee. Keeping things inside the boundary is merely a choice to confine computation there — one that no longer holds the moment you need to borrow external compute or collaborate with another organization. What matters is not the location, but whether the party entrusted with the computation can see the data.

Private AI's criterion ultimately converges to a single line. While AI works on my data, the party entrusted with the computation — even the model operator — must not be able to see that data. And at the center of the path that fully satisfies this criterion stands fully homomorphic encryption (FHE).

2. Why now: the last bottleneck in AI adoption

The usefulness of generative AI is already proven. And yet adoption is slowest precisely where the value is greatest — finance, healthcare, the public sector, legal, and the core technical units of manufacturing. The reason is not model capability; it is data. Three pressures are bearing down at once.

Data has become the prompt. If the early worry was "will my data be used for training," the reality of the agentic-AI era is more direct. The more useful an AI is, the more context it demands — and that context is, in the end, internal documents, customer records, inboxes, and databases. A structure in which an organization's entire body of knowledge becomes inference input and leaves the boundary has become the norm. Indeed, many companies at home and abroad have adopted policies banning internal information from external chatbots, and in 2023 Italy's data-protection authority temporarily suspended a generative AI service over privacy concerns.

Regulation is squeezing from both sides. On one side, GDPR, the EU AI Act, and national privacy laws are strengthening data-protection obligations; on the other, rules requiring data cooperation between institutions keep multiplying — like the information-sharing provisions of the EU Anti-Money Laundering Regulation (AMLR). The double demand — "share, but don't expose" — is a problem the existing playbook struggles to solve.

Agents demand permissions. Agentic AI presupposes access to inboxes, calendars, internal systems, and payment methods. The broader the automation, the larger the exposure surface and the attack surface grow together. You need a way to hand the agent the keys without showing it the inside of the vault.

So for a long time, many organizations' options narrowed to two: don't use it, or use it and accept the risk. Private AI is the third option.

3. What leaks: the exposure points in the AI pipeline

Take the AI pipeline apart from a security standpoint, and there are more points where data is exposed as plaintext than you might expect.

StageData in motionWho can see it
Prompts and conversationsThe questions themselves, attached documents, chat historyModel operator, cloud infrastructure provider
RAG retrievalThe entire internal document store and its embeddingsSearch and vector-DB operators (research keeps showing that much of the original text can be recovered from embeddings)
Training and fine-tuningTraining dataWhoever operates the training + possible later extraction via model memorization
Agent tool callsContents and credentials of email, DBs, and APIsThe agent platform and intermediary infrastructure
Logs and monitoringPlaintext copies of inputs and outputsThe entire ops and security toolchain

What is worth noting is that none of these is a problem caused by "failing to encrypt." Encryption in transit (TLS) and at rest is already the industry standard. The problem is the third stage. The moment data is computed on (in use), it all reverts to plaintext. AI is, in essence, computation — so the privacy problem of the AI era converges on this "data in use" problem. Private AI's approaches are all attempts to fill this gap, each in a different way.

4. The first answer, bringing the model to the data: private deployment

The most intuitive answer is not to send the data at all. Run an open-source model on your own servers or a private cloud, and operate AI inside the organization's boundary. Because the data never crosses the boundary, regulatory compliance is straightforward, and many organizations do in fact take this path.

But three limitations come attached. First, the burden of build and operations: GPU infrastructure, MLOps staffing, and model-update cycles all have to be shouldered in-house, and the cloud's elasticity of scaling up and down with demand has to be given up. Second, being confined within the boundary: analysis and collaboration that require combining data with other institutions are impossible from the start. Third, seen from the model side, there is a trust problem running the other way. Model providers are reluctant to hand their models over to customer infrastructure for fear of weight leakage. A standoff forms — the data owner cannot release the data, the model owner cannot release the model — and simply bringing a good model inside the boundary becomes a negotiation problem in itself.

Private deployment is a valid option. But the moment "computation beyond the organization's boundary" becomes necessary — borrowing external compute, or combining data with another institution for analysis — it is powerless. Hence the next question: can we compute outside the boundary, without being seen?

5. The second answer, moving the trust: TEE, MPC, federated learning, and differential privacy

The family of technologies for "protection during computation" is called privacy-enhancing technologies (PETs). Each is an excellent tool. But hold one question fixed — when you use this technology, what must you trust? — and a common structure emerges. They do not remove trust; they change its target or limit its coverage.

TEE (Trusted Execution Environment, confidential computing). It creates isolated regions inside CPUs and GPUs and encrypts memory, so that not even the cloud provider can look inside. Its great advantages are speed nearly identical to plaintext computation and commercial availability today on major clouds and the latest GPUs. What you must trust comes down to two things, and they are problems of different natures. One is a trust-model problem: the root of the isolation and its attestation lies with the chip maker, so trust has not disappeared — it has moved from the cloud provider to the chip maker. The other is an implementation problem: what keeps the isolation intact is not mathematics but hardware and firmware implementation, so the completeness of that implementation must be trusted as well — and this is exactly where side-channel vulnerabilities have been steadily reported in academic research, along with demonstrated cases.

MPC (secure multi-party computation). Data is mathematically split into shares that multiple servers compute on separately. What you must trust is the assumption that the servers do not collude. A single share reveals nothing, but if the servers join hands, the original can be reconstructed. On top of that comes a structural burden. It is an interactive protocol with communication in every round of computation: BOLT, the representative MPC-hybrid study cited in the THOR paper, reported about 25.74 GB of communication for a single BERT-base inference. Homomorphic encryption, by contrast, is a non-interactive structure — send the ciphertext, receive the result — so this kind of round-trip communication during inference simply does not exist.

Federated learning. It moves model updates instead of data, avoiding data concentration at the "training stage." What you must trust are the participating institutions and the aggregation server, and its coverage is limited to the movement of training data. Input privacy at inference time is not addressed at all.

Differential privacy. It adds mathematical noise to results, hiding individuals inside aggregates. Less a matter of whom to trust than of limited coverage: it protects the "output" at the point of statistical release or model publication, while the "input" during computation stays plaintext — with a trade-off of accuracy surrendered in proportion to the noise.

These technologies are not mutually exclusive, and in practice they are often combined. But they share one thing: somewhere, there remains "a party you must still trust" or "a stretch that goes unprotected."

6. The third answer, removing the trust: homomorphic encryption

Fully homomorphic encryption (FHE) changes the question. Not whom to trust, but can we make it so that no one needs to be trusted? Because data can be computed on while it stays encrypted, the server handles nothing but ciphertext from start to finish. Inputs, intermediate values, results — none of them ever exists as plaintext. There is simply no party that can see.

The basis of this guarantee is neither operating policy, nor hardware isolation, nor a non-collusion assumption, but the mathematical hardness of lattice problems (Ring-LWE). Resistance to quantum computers follows from the same mathematics. And, as THOR demonstrated, a non-interactive structure is possible — the operation reduces to a simple flow of sending ciphertext once and receiving the result once.

Put the approaches side by side in one table and the differences become clear.

ApproachWhat you must trustCoverageCharacteristics
Private deploymentYour own infrastructure and insidersPrevents data from moving outsideBuild/ops burden; limits collaboration across institutions
TEEChip maker and firmwareDuring computation (hardware isolation)Excellent speed; documented side-channel attacks
MPCNon-collusion among serversDuring computation (distributed)Interactive; heavy communication
Federated learningParticipating institutions and the aggregation serverPrevents training-data movementDoes not protect inference
Differential privacy(Soundness of the output design)At the point results are releasedDoes not protect inputs; accuracy trade-off
Homomorphic encryption (FHE)Mathematics (lattice hardness) aloneAll stages — in transit, at rest, and in useStrongest guarantee; the challenge is computational cost

In practical terms, homomorphic encryption's only price has been computational cost. So the question of the past 15 years was never "does it work?" but "when does it become practical?" TEEs and homomorphic encryption differ in horizon more than they compete. TEEs provide protection during computation at today's performance; homomorphic encryption points to the terminus where no trust is needed at all. And the distance to that terminus has shrunk dramatically over the past few years. That is the story of the next section.

7. Proof of practicality: what THOR and GL changed

The evidence that FHE-based Private AI is more than a slogan comes at two levels: the algorithm and the scheme.

The algorithm level: THOR. Jointly developed by DESILO and Professor Miran Kim's research team at Hanyang University and presented at ACM CCS 2025, the top-tier venue in computer security, THOR redesigned matrix multiplication and nonlinear approximation on top of the existing fourth-generation scheme (CKKS), running the full encrypted inference of a BERT-base model on a single GPU in 10 minutes as of the original paper, since cut to around 2 minutes through continued optimization. The previous work in the same non-interactive style took 2.7 hours for that task, and accuracy differs from the plaintext model by only about 1 percentage point. "Encrypted LLM inference" came down from hours to minutes.

The scheme level: GL. The fifth-generation GL Scheme, co-invented by FHE's founder Craig Gentry and DESILO Chief Scientist Yongwoo Lee, redesigned homomorphic encryption from the ground up for matrix multiplication — the operation that dominates AI computation. An encrypted matrix multiplication reduces to four plaintext matrix multiplications, and the overhead of key switching is bounded within at most four times the cost of element-wise multiplication. Because most of the computation time becomes "plaintext matrix multiplication you would have to do anyway," encrypted computation speeds up in step with GPUs and dedicated hardware.

Then a third axis joins in: hardware. Research on FHE-specific accelerators and GPU optimization continues, and a scheme like GL — restructured around plaintext matrix multiplication — structurally absorbs the gains of that acceleration. With algorithms, schemes, and hardware all moving at once, homomorphic encryption's cost curve is falling faster than any single axis alone.

The details are covered in Understanding THOR and Understanding the GL Scheme, respectively.

8. What you can build: from Secure RAG to agentic AI

FHE-based Private AI is already taking the shape of concrete products.

Secure RAG. A structure that stores an organization's documents encrypted and keeps the question encrypted all the way through retrieval and generation. The RAG exposure points from Section 3 — the document store, the embeddings, the queries — move wholesale inside the ciphertext. Connecting an organization's knowledge to AI while that knowledge remains visible to no one: the most practical first form of Private AI.

Private assistant. An AI assistant that uses personal and organizational context (email, calendar, documents) while it stays encrypted. Instead of trading privacy for convenience, the direction is to have both.

Data clean room. Institutions that cannot show one another their data combine it in ciphertext form to perform joint analysis and joint modeling. It is the direct answer to the "share but don't expose" double demand (Section 2), as in the joint detection of money laundering across financial institutions — and DESILO's DESILO DCR takes this form.

A safety layer for agentic AI. If the data and credentials an agent handles are kept encrypted, you can grant the agent authority without growing the exposure surface. The role of cryptography in agent-era data security is an area where serious discussion has only just begun.

DESILO is building this stack from the bottom up: the FHE library DESILO FHE Library, which includes the fifth-generation GL Scheme; DESILO DCR for combined analysis across institutions; and DESILO Private AI, which includes Secure RAG. From scheme (GL) and algorithm (THOR) to library and product, companies that build every layer of FHE-based Private AI themselves are rare anywhere in the world.

9. Frequently asked questions (FAQ)

Q1. Does Private AI mean on-premises AI?

No. It is a question of guarantee, not of deployment location. On-premises installation is merely a choice to confine computation within the boundary, and it no longer holds the moment external compute or collaboration with other institutions becomes necessary. Private AI's criterion is "can the party entrusted with the computation see the data" — and that is decided by architecture, not deployment model.

Q2. I've heard homomorphic encryption is too slow to use. Is that true?

Search, statistics, and structured-data analytics workloads have entered the phase of verification for practical applications, and for LLM inference THOR brought the unit of time from hours down to minutes. With fifth-generation technology like the GL Scheme, which changed the mathematical structure itself, plus hardware acceleration, the gap keeps closing.

Q3. If TEEs exist, why bother with homomorphic encryption?

They are closer to complements than competitors. A TEE provides protection during computation at today's performance but requires trusting the chip maker and the implementation; homomorphic encryption guarantees security through mathematics alone, with no one to trust. Choose and combine according to the required level of assurance and the workload — but for data where no trust can be permitted at all, the terminus is homomorphic encryption.

Q4. Doesn't encryption degrade AI quality?

THOR's results are the answer. On the GLUE benchmark, the difference from the plaintext model was about 1 percentage point. Most of that difference comes not from the homomorphic operations themselves, but from the error that arises in approximating with polynomials the functions homomorphic encryption does not directly support — such as Softmax and normalization, which are widely used in AI models and involve division.

Q5. Can training (fine-tuning) also run encrypted?

This is an area of ongoing research. The current center of practical use is inference, retrieval, and analytics; training in encrypted form is the next challenge. Foundations like the GL Scheme, which accelerates matrix operations at the root, are narrowing this gap as well.

10. References

Tech & Insight — The Homomorphic Encryption & Private AI Series (4 parts, complete)