Here are the key factors you need to evaluate before picking an AI vendor

Yogi Schulz

For interview requests, click here

As generative AI adoption surges, many organizations are upgrading their products and services with large language model (LLM) features. But choosing the right LLM and supporting software is no small task. Here’s what you and your chief information officer should consider to make an informed choice.

Step 1: Choosing an LLM vendor

Most organizations license LLMs and supporting software rather than building them in-house. Selecting the right vendor requires careful evaluation.

To reduce risks, consider these criteria:

  • Track record: Does the vendor have experience with similar clients?
  • Cybersecurity: Are their software architecture and defences robust?
  • Financial stability: Can the vendor support your needs long-term?
  • Defect management: How do they address software issues?
  • Data handling: Are storage and deletion practices secure?
  • Third-party audits: Have they been recently assessed?
Recommended
Breakthrough algorithm equips AI systems to continuously adapt and learn
How ChatGPT can revolutionize oil and gas work processes
Chat GPT a wake-up call for business leaders everywhere

MORE BUSINESS MANAGEMENT TIPS

Because many LLM vendors are new, developing a contingency plan is prudent in case the chosen vendor ceases operations.

Step 2: Data usage agreement

A clear data usage agreement protects sensitive information and reduces breach risks. Key considerations include:

  • Data rights: Define access, storage, and protection responsibilities.
  • Incident response: Establish and test plans for data breaches.
  • Compliance: Ensure agreements align with your privacy policies and regulations.

Step 3: Secure data transmission

When transferring data, prioritize security by implementing:

  • Encryption protocols for data in transit.
  • Secure file transfer methods.
  • Data loss prevention mechanisms.

These measures reduce the risk of data breaches and comply with privacy standards.

Step 4: Selecting the right LLM software

LLM software selection requires careful evaluation. Many options have limited track records, so consider these factors:

Key Criteria

  • Functionality: Does the software meet your needs, such as domain adaptability?
  • Accuracy: Is output clear, grammatically correct, and unbiased?
  • Speed: Can the software deliver results promptly?
  • Scalability: Will it support growth without performance issues?
  • Support: Is vendor support reliable, and is there an active user community?
  • Cost: Are operating expenses sustainable?

A detailed questionnaire will help objectively compare software options.

Step 5: Protecting sensitive data

Minimize risks by reducing the sensitive data shared with the LLM. Strategies include:

  • Anonymization: Use anonymized or synthetic datasets for testing.
  • Data limits: Restrict access to personally identifiable information.
  • Resource management: Delete unused virtual machines and suspend idle ones.

Step 6: Managing software risks

LLM software evolves rapidly, often with frequent updates. To manage risks:

  • Budget time and resources for testing.
  • Regularly review release notes and plan for updates.
  • Maintain rollback procedures to revert to earlier versions if needed.
  • Only move software to production after user acceptance testing confirms reliability.

Step 7: Avoiding customization

Customizing LLM software is costly and often problematic. Instead:

  • Use software as designed, adjusting built-in settings where necessary.
  • Avoid modifying source code to minimize update challenges.
  • Advocate for needed features by participating in vendor advisory groups.

Focusing on configuration rather than customization ensures smoother updates and reduces long-term costs.

Choosing the right LLM and vendor is critical for organizations adopting generative AI. Thorough vendor evaluation, secure data practices, and cautious software management reduce risks and set the stage for success.

By making thoughtful choices, your organization can integrate AI tools that enhance products and services responsibly and effectively.

Yogi Schulz has over 40 years of information technology experience in various industries, including extensive work in the petroleum industry. He manages projects that arise from changes in business requirements, the need to leverage technology opportunities, and mergers. His specialties include IT strategy, web strategy, and project management.

Explore more on Artificial intelligence

© Troy Media

Troy Media is committed to empowering Canadian community news outlets by providing independent, insightful analysis and commentary. Our mission is to support local media in building an informed and engaged public by delivering reliable content that strengthens community connections, enriches national conversations, and helps Canadians learn from and understand each other better.