Loading insights
Loading insight article
Insights / Production Engineering
Every prompt typed into a public AI tool is a document leaving your firm's walls. Here is a step-by-step guide to deploying private AI on iManage.It covers architecture, hardware, permissions, and rollout.Client documents stay on your network.
Every prompt you type into public AI tool, is a document leaving your firm's walls. For most industries, that's minor inconvenience. For law firms, it's a potential breach of privilege. That's the tension driving a quiet shift across legal IT departments right now: firms want the speed of AI-assisted document review and research but they can't risk client data sitting on servers they don't control, governed by terms of service they didn't write. A cloud AI tool that logs, retains or trains on a client contract isn't a hypothetical risk, it's a Rule 1.6 problem waiting to happen.
This is where iManage AI integration built for private deployment comes in. Instead of routing documents through third-party AI infrastructure, firms are connecting private AI directly to their own iManage environment, no data leaving the building, no exceptions. If you need private AI for law firms that doesn’t compromise client trust, this tutorial walks you through architecture, setup and governance, step by step.
Attorney-client privilege does not take a holiday. Rule 1.6 of ABA Model Rules requires lawyers to protect all information relating to representation of client regardless of its source. Unless the client gives informed consent and ABA Formal Opinion 512 makes clear this duty applies directly to generative AI tools. Under Rule 1.6, the risk of inputting client information into an AI tool must be evaluated before it happens and for self-learning tools, informed client consent must be obtained first.
The stakes are not theoretical. BakerHostetler's 2026 Data Security Incident Response Report found law firm breach activity has nearly doubled, with 56% of breached firms losing sensitive client information and average breach cost has climbed to $5.08 million. Notably, quarter of incidents came in through a third-party vendor exactly the exposure that a cloud AI tool adds.
On-prem legal AI removes that variable entirely. For firms that can't get explicit client consent for cloud processing, on-prem legal AI is only defensible path. That satisfies ABA Model Rule 1.6(c) and avoids the "most dangerous AI tool in your firm is the one that lives outside it" trap.
Features like Ask iManage, AI Enrichment and Mailbox Assistant provide real productivity gains by connecting your documents to cloud hosted large language models. Ask iManage answers questions based on your firm's documents, AI Enrichment auto classifies and extracts from contracts and Mailbox Assistant files emails automatically. All three are convenient and quick and all three inherit iManage's existing security model.
While genuinely useful, it isn't the same thing as iManage AI integration built for a private deployment. The inference layer still runs on iManage managed cloud servers, not inside your own network. For firms that need zero external inference, not just inherited permissions, native features fall short.
Scroll horizontally to view all columns.
| Native iManage AI | Private Deployment | |
| Where inference runs | iManage-managed cloud | Your own servers / sovereign environment |
| Setup effort | None, built in | Requires architecture, hardware, integration work |
| Data boundary | Trusts iManage's cloud security model | Data never leaves your network |
| Best for | Fast adoption, standard workflows | Strict privilege, ethical wall or client-consent requirements |
There is no one right answer here, it’s a governance decision not just technical one. Cloud deployment wins on speed and ease, on-premise wins on data control and cost predictability and by 2026 roughly 70% of organizations ended up somewhere in between with a hybrid model. Choosing the right deployment model for iManage AI integration depends on your risk tolerance, IT capacity and client consent posture. Below is a practical comparison across the most relevant criteria.
Scroll horizontally to view all columns.
| Cloud (iManage native) | Hybrid (Private Core + Cloud Burst) | Private / On-Prem | |
| Data boundary | Documents processed on external LLM APIs (e.g., Google Gemini, Microsoft Azure OpenAI). | Sensitive matters on-prem; non-confidential workflows (e.g., marketing, training) on cloud APIs. | All data stays on firm-owned servers or sovereign cloud; no external egress. |
| Control | Limited to iManage feature set; no model or architecture control. | Moderate control; can tune private layer while usng cloud scale for low-risk tasks. | Full control over model selection, vector store, permissions enforcement, and audit logs. |
| Compliance burden | Low operational burden; rely on vendor's SOC 2, ISO 27001 and data processing agreements. | Medium: must classify data rigorously and maintain dual governance policies. Highest upfront effort, easiest to defend | High: firm must manage security patches, model updates and ethical wall testing under Rule 1.6 |
| Cost tier | Usage-based, low upfront, can scale unpredictably | Mixed, fixed cost for on-prem tier + variable cloud spend | High upfront (hardware/setup), lower marginal cost at volume |
| Best fit | Firms prioritizing fast rollout, standard matters | Firms with mixed sensitivity across matter types |
Before deploying a private AI layer on top of iManage, confirm that your environment meets these technical and administrative requirements. Skipping any of these will block integration or compromise security.
Without the right iManage AI integration prerequisites, especially API access and hardware, you cannot enforce permissions or keep data on-prem. Firms that underestimate on-prem legal AI hardware needs often find themselves facing latency or compliance failures post-deployment.
This workflow walks you through a private AI deployment that keeps all document content on your infrastructure while enabling AI-powered search, summarization and drafting on top of iManage.
(iManage's OAuth-based app registration process is also used by third-party tools like Forcepoint DSPM, see their registration walkthrough for a visual reference of the Control Center screens, though the permission scopes differ for an AI integration.)
Obtain OAuth2 credentials so your private AI server can authenticate to iManage Work APIs without exposing user passwords.
Ensure no document embeddings or prompts leave your firm's network.
Run a large language model entirely on your hardware, with no external API calls.
Actions:
Connect your private LLM to iManage documents via retrieval-augmented generation (RAG), enforcing permissions at query time. colabcontent+1
Actions:
Validate that permissions, retrieval and model outputs work correctly before exposing sensitive matters.
Don't skip this step to save time; permission leaks are the single most damaging failure mode for legal AI.
Minimize risk and gather feedback before firm-wide deployment.
Your private AI layer must enforce the same access controls as iManage Work otherwise, you've created a privilege escalation vector. Run through this checklist before go-live to validate that your deployment respects ethical walls and per-matter restrictions.
This is the layer that turns iManage AI integration from "documents stay on our servers" into something you can actually defend. Infrastructure alone doesn't preserve ethical walls; enforced, tested access control does.
Sizing depends on firm size and concurrent users, not just document volume. Treat these numbers as a planning baseline, not a quote; always validate against a proof-of-concept on your actual workload before committing budget. Here's an estimated starting point for each tier:
Scroll horizontally to view all columns.
| Firm Size | GPU Configuration | RAM | Storage (NVMe SSD) | Annual Cost Estimate |
| Small (10–50 attorneys) | 1× NVIDIA A10 (24GB) or A40 (48GB) | 128GB | 2TB NVMe | $15,000–$25,000 |
| Midsize (50–200 attorneys) | 2× NVIDIA A100 (40GB each) | 256GB | 4TB | $40,000–$70,000 |
| Large (200+ attorneys) | 4× NVIDIA A100 (80GB each) or H100 cluster | 512GB+ | 8TB+ | $120,000–$250,000+ |
Below given mistakes turn private AI into a governance liability. Address them in your pilot phase, not post-deployment.
Scroll horizontally to view all columns.
| Mistakes | How to Fix them |
| Skipping Human-in-the-Loop Letting the AI layer auto-generate client-facing drafts or legal conclusions without attorney review. | Require human approval for all outputs that touch matter strategy, client communications, or filing documents. Log reviewer ID and timestamp for audit. |
| Underestimating Data Cleanup Connecting AI to a messy iManage instance with duplicate files, outdated versions, and misclassified matters. | Run a pre-deployment cleanup sprint: archive obsolete matters, enforce consistent metadata schemas, and tag sensitive matters for AI exclusion. |
| Treating Integration as an Afterthought Building the AI layer first, then trying to bolt on iManage permissions and ethical walls later. | Design your RAG pipeline around iManage's OAuth2 and ACL model from day one. Test permission enforcement before onboarding any users. |
Private AI on iManage isn't something you set up once and forget, it's an ongoing commitment to keeping client data safe while still using AI to work faster. The steps in this guide get your documents flowing through your own servers instead of cloud but that's really just half the job. The other half is making sure permissions are enforced, every query is logged and a real lawyer still checks the AI's work before it reaches a client or a court. Private deployment solves the confidentiality problem but it doesn't solve the accuracy problem, those are two different things, and firms that forget this are the ones who end up in the news for the wrong reasons. Get the setup right from the start and everything after that gets a lot easier.
Ready to get started? Begin with Step 1: register your client app in iManage Control Center and aim to have your first test running within the week.
iManage AI integration connects AI tools to documents and knowledge stored in iManage. It allows lawyers to search, summarize, analyze, and retrieve information using natural-language queries.
Yes. Law firms can connect iManage to a private AI environment through APIs or connectors, allowing AI to work with authorized documents without necessarily sending them to a public AI service.
Private AI runs in a controlled environment, which can be cloud-based or on-premises. On-prem legal AI specifically runs on infrastructure controlled and hosted by the law firm.
Hardware depends on the AI model, number of users, document volume, and workload not simply firm size. Smaller deployments may need one professional GPU, while larger workloads may require multiple data-center GPUs.
5. Does private AI eliminate AI hallucinations in legal work?
No. Private deployment helps control where data is processed but does not guarantee accurate AI outputs. Lawyers should still verify AI-generated answers, citations, and summaries before relying on them.
Most firms move through registration, hardware setup, RAG pipeline build, and permission testing in 4–6 weeks, with practice-group rollout following over the next 4–8 weeks depending on firm size and IT resourcing.
Tell me about the system or workflow you are working on.
30-minute intro call on Calendly
| Firms where privileged/high-conflict matters dominate |
