2026.08.24
生成AI・LLMアーキテクチャ
LLMs have taken the world by storm and are being advertised as the theory of everything. One that knows the solution to every problem; just throw your query to AI and get the solution. However, it is interesting to look into how classic Machine Learning approaches can be more efficient and how sometimes less is more!
The point is not to argue that LLMs are overrated or that classical Machine Learning is superior. Rather, it is to recognise that different problems require different tools. LLMs have made remarkable progress in handling unstructured information, reasoning, and language-based tasks, but many real-world problems are fundamentally about prediction, optimisation, and working within strict constraints. In these cases, a smaller, specialised model or even a classical optimisation algorithm can be faster, cheaper, more reliable, and easier to evaluate.
The real question, therefore, is not “How can we use an LLM?” but “What is the right tool for this problem?”
The rise of Large Language Models has changed the way we think about solving problems. Given the right prompt, an LLM can write code, analyze documents, generate explanations and even interact with external solutions to almost every problem.
But versatility does not mean suitability.
Consider a simple question from the logistics industry:
This is not fundamentally a language problem. It is a prediction problem.
The answer depends on structured signals such as historical demand, location, time of day, day of the week, seasonality, holidays, weather, and other operational factors. The objective is not to generate a convincing answer. It is to produce an accurate prediction that can be measured, evaluated, and continuously improved.
For example, for a trend prediction problem like hospital bed demand a Support Vector Regressor may work much better than just a simple LLM since it is deterministic and is designed for that kind of input.
In our previous example we talked about hospital bed demand prediction. This may be a single project but it is important to notice that prediction work is still very relevant in business environments. A simple model predicting numbers like sales and revenue can go a long way in planning and strategising for optimal results.
The important point is that these are fundamentally structured prediction problems. The model is given historical data and is expected to produce a measurable output, such as expected sales next month or the probability that a customer will churn. The quality of the model can be evaluated directly using metrics such as MAE, RMSE, precision, recall or AUC.
While prediction problems ask us to estimate what is likely to happen, scheduling problems ask a different question: given a set of constraints, what is the best way to allocate limited resources?
Consider something as simple as scheduling employees in a hospital. There may be hundreds of employees, multiple shifts, different skill requirements, maximum working hours, employee availability, leave requests, and minimum staffing requirements. The objective is to create a schedule that satisfies these constraints while minimising costs or maximising efficiency.
The same problem appears across industries. Airlines need to assign crews to flights, factories need to assign machines to production runs, and logistics companies need to assign drivers and vehicles to delivery routes. In each case, the number of possible combinations can grow extremely quickly as the size of the problem increases.
An LLM could be given the requirements and asked to produce a schedule. It might even produce one that looks perfectly reasonable. However, looking reasonable is not the same as being optimal or even valid. A single overlooked constraint, such as a driver exceeding their maximum working hours or a machine being assigned to two jobs at the same time, can make the entire schedule infeasible.
This highlights an important distinction between language generation and optimisation. An LLM is fundamentally designed to generate a likely sequence of tokens based on its learned patterns. An optimisation algorithm, on the other hand, can be explicitly designed around the constraints and objective of the problem.
The appeal of an LLM is its generality. A single model can summarise a document, write code, answer questions, analyse data, and interact with users. Compared to building a separate system for every individual task, this seems like an obvious advantage.
However, generalisation comes with a cost. In a production environment, the question is not simply whether a model can perform a task, but whether it can perform that task reliably, efficiently, and economically at scale.
The direct consequence of using LLMs irresponsibly may cause problems such as bloated size and costs. Say a company needs to make millions of predictions every day, even a small difference in inference cost can be too great to bear.
Add to that the simple cost of hardware if an in-house LLM is desired; with the climbing costs of GPUs and RAMs it becomes impractical to think of controlled cost approaches for personal or corporate-wide LLMs.
The problem is amplified when we realise that to do the same task a good LLM model may take much longer to give responses that are much less reliable. A focussed approach can aid in generating reliable responses.
Another cost to consider when adopting LLMs is:
A large portion of enterprise LLM usage happens through APIs provided by external model providers. While these services can be secure and offer enterprise-grade controls, they still introduce an additional layer between a company’s data and the model processing that data. For businesses dealing with sensitive information, this creates questions around data residency, retention, access controls, compliance, and third-party risk.
Consider a hospital using an AI system to analyse patient records, or a financial institution processing transaction data. Sending this information to an external model may create requirements around consent, regulatory compliance, encryption, auditing, and contractual data handling. The problem is no longer simply whether the model produces an accurate answer. The organisation must also be able to demonstrate that the data is being handled appropriately.
It is also important to note that data going through APIs to big LLM players runs the risk of that data being used as training data for their models. That data may end up being outputted as responses for other prompts increasing the risk of data leakage.
Moreover LLMs, still being a newer technology, have introduced various new challenges which exacerbates the risks of data leakage to people with malicious intent.
Generality can also come at the cost of speed.
Many business systems operate under strict latency requirements. Fraud detection may need to happen before a transaction is approved. Anomaly detection may need to happen as sensor data arrives. A logistics system may need to recalculate thousands of routes as conditions change.
In these situations, a specialised model that produces a prediction in milliseconds can be more useful than a much larger model that takes significantly longer to respond.
Another hidden cost is evaluation.
For a traditional prediction model, we can define a target and measure performance using metrics such as MAE, RMSE, precision, recall, or AUC. This makes it relatively straightforward to determine whether a new model is better than the previous one.
With LLMs, evaluation can become much more complicated when the output is open-ended. Is one explanation better than another? Is a generated summary complete? Did the model miss an important detail? Did it introduce information that was not present in the source?
This does not make LLMs unsuitable. It simply means that the cost of proving that they work correctly can be considerably higher.
Perhaps the most overlooked cost of generalisation is that businesses can end up paying for capabilities they never use.
If a system only needs to classify transactions, predict demand, or detect anomalies, the ability to write an essay or explain a concept has little operational value.
This is similar to using a Swiss Army knife to perform a task that requires only a screwdriver. The additional tools may be impressive, but they do not necessarily make the job better.
The goal of production AI should therefore not be to maximise the capabilities of every component. It should be to minimise the complexity required to reliably solve the problem.
Generality is valuable when the problem itself is general. When the problem is narrow, however, specialisation can be a competitive advantage.
A primary divergence between conventional logic and LLM-centric architectures lies in the level of predictability one can anticipate from their respective outputs.
A system is defined as deterministic when identical inputs, processed under stable conditions, yield an unvarying result. Consider a basic calculation for an order total; provided the rules and data are static, the outcome remains constant across every execution.
It is critical, however, to separate standard software from the realms of Machine Learning and optimization. Classical ML frameworks and solver algorithms are not always strictly deterministic in practice. Their performance often fluctuates based on algorithmic choices, specific configurations, random seeds, or inference parameters. Yet, the strength of these specialised systems resides in their capacity to have objectives, boundaries, and performance benchmarks defined with extreme precision.
LLMs function on a different premise. They are inherently probabilistic, synthesising responses by identifying patterns from their training data and the immediate prompt context. Consequently, even when tackling the same objective, their generated content may shift depending on the specific model or its sampling settings.
Such variance is not a flaw by default. On the contrary, this fluidity is exactly why LLMs excel at linguistic tasks. There are countless legitimate ways to condense a report, clarify a concept, or engage with a client, and we rarely demand a verbatim repetition of the same phrasing.
The complication occurs when this inherent elasticity is forced upon tasks where uniformity, rigid constraints, and forecastable outcomes are far more critical than producing a creative response. Take, for instance, a logistics engine that must verify a driver stays within legal service hours, or a financial platform that must execute a fixed interest formula. In these scenarios, where a solution must satisfy a strict set of parameters, the system must be rigorously governed, ensuring its operations are both measurable and verifiable.
This is the domain where specialized models and legacy software provide a significant edge. While they may not be purely deterministic by nature, they are far more amenable to being tightly restricted, benchmarked, and replicated within a controlled framework. Still, determinism is not an absolute virtue. A support bot might perform better by offering varied responses to keep a conversation natural. Systems designed for human interaction often prioritize adaptability over identical outputs. Ultimately, the goal is not to favor deterministic logic over probabilistic models. The objective is to align the degree of control and predictability with the specific demands of the problem at hand.
When a single correct answer exists or strict boundaries must be maintained, predictability is a vital feature. Conversely, when several solutions are viable, flexibility itself becomes the advantage. However, determinism should not be treated as universally better. A customer support assistant may actually benefit from producing different responses depending on the conversation. A human-facing system often needs flexibility rather than identical outputs.
This distinction becomes even more important when the prediction feeds into another operational decision. If a logistics company underestimates tomorrow’s demand, it may not have enough drivers available. If it overestimates demand, it may allocate excess capacity and increase costs unnecessarily.
In such cases, the question should not be “Can an LLM solve this?”
It should be:
And sometimes, that tool is not an LLM at all. In these cases it is important to actually employ research and invest time in coming up with the best solution that saves not only the cost but also the hardware and time overhead.
