Is Machine Learning Worth Using? A Fair Comparison with Simple Methods

A new model reduces prediction error to zero and changes the selected stock. Before replacing the old method, you inspect the cost ledger: both accounts end with exactly the same money. Why did greater prediction accuracy produce no additional profit?

Prediction, selection, and execution are connected by rules. Fit one line and one simple tree, then follow each through the same cash-account conditions. You need not adopt machine learning afterward. Explaining what it changed—and what it failed to add—is a useful comparison result.

Distinguish the jobs being assigned to AI

You will fix training data, labels, and two candidates; check errors, selection, and fee-inclusive results; and repeat the original and one controlled variant in the offline teaching aid. Save a T19 record of incremental model value, including a result of none.

An economic strategy first asks whether a price-related feature known at the decision time can help select stocks for the next holding period. A prediction algorithm transforms features into numerical estimates. Selection determines which assets and quantities to hold. Execution handles orders, fills, cash, and exceptions. A prediction does not establish a purchase.

An LLM assistant is another layer. It can organize evidence, help implement a model, and explain outputs; it is neither the factual feature source nor the fitted OLS or tree in this exercise. Generated code does not remove the need to check training boundaries, data permissions, and accounts. A fluent explanation cannot replace an actual model output.

Freeze a task small enough to inspect

Standalone C9-90-v1 is synthetic and does not replace C7 momentum inputs or real C9 research. The sole feature, x, is an abstract predetermined price score, measured in score units—not a share price or future return. Four training assets have earlier T0 features. Their gross labels come from buying at T1 open and selling at T2 open; T2 precedes the later D0 decision, so all training labels are then available.

Later assets A, B, C, and D again have x values 1, 2, 3, and 4 at D0, with complete identities and eligibility supplied. Do not train again after D0. Buy no earlier than D1 open and exit at D2 open. The holding periods do not overlap. Neither randomly mixing rows from one date nor using D2 returns as D0 features is allowed.

Compare only two registered candidates: ordinary least squares with an intercept, and a squared-error regression tree with maximum depth 1 and at least two rows per leaf. No new features, depth search, weight search, scaling, or imputation enter this example. When real work needs preprocessing, learn its settings only from permitted training data, then apply them to later data. Test observations cannot choose the model.[1]

The authors constructed and know every label. Initially hiding later outcomes guides reading; it does not supply an unknown holdout. Four training rows and one later cross-section cannot establish stable market usefulness, regime coverage, or protection from trial selection.

Fit a line and a single split

Ordinary least squares, OLS, chooses a line minimizing the sum of squared training prediction errors. The official documentation explains that form and fitting objective.[2] Here, mean x is 2.5 and mean return 0.02. The fitted slope is 0.016 and intercept −0.02: predict −0.02 + 0.016x.

For x = 3, the prediction is 0.028, or 2.8%; x = 4 predicts 4.4%. These are return estimates, not probabilities. A negative prediction therefore does not mean a negative probability. A continuous linear change may or may not suit a real relationship.

A regression tree splits observations by feature conditions and predicts a value within each leaf. Official documentation describes piecewise-constant predictions and depth and leaf constraints.[3] With four rows and two required in each leaf, this example can split only between 2 and 3: threshold 2.5. The left pair averages 0%; the right pair 4%.

The tree learned a low-group/high-group step. It did not understand companies, identify causation, or discover a market mechanism. Beyond the training feature range, a line and tree will still output numbers according to their forms; that does not demonstrate coverage of those new conditions.

Inspect predictions before choosing a winner

Apply frozen models to later assets, then reveal original later gross labels 0%, 0%, 4%, and 4%.

Mean absolute error, MAE, averages the absolute prediction–label differences. OLS errors are 0.4, 1.2, 1.2, and 0.4 percentage points, averaging 0.8 points. Tree errors are all zero. If you also retain mean squared error, MSE, use decimal returns: OLS 0.00008, tree 0. Squared decimal error is not a return percentage.

Select the highest-predicted 20%, rounding the number of assets up: four assets produce one selection. Predictions within 1e−12 tie, resolved by ascending stable asset ID. OLS selects D. The tree ties C and D at 4%, so it selects C. The tree did not predict C would outperform D; the registered tie rule chose it.

C and D both gain 4% gross in the original. Prediction error genuinely falls, but the selected assets happen to earn the same return. A net-return claim still requires the cost ledger. Zero error cannot be relabeled pure alpha.

Put both models through identical cash accounts

Each starts with $10,000 cash, buys completely at $100 on D1, and sells completely at $104 on D2. This independent comparison charges 5 basis points commission on each side, with no slippage, taxes, distributions, or interest. It is not C2's commission-plus-directional-slippage baseline.

Calculate a fee-inclusive purchase, then round down to one-millionth of a share: 99.950024 shares. Purchase notional is $9,995.0024, buying commission $4.9975012, and remaining cash $0.0000988. Buying 100 shares and accepting negative cash would violate the account.

Sale proceeds before commission are $10,394.802496, and selling commission $5.197401248. Add the small remaining cash after deducting the selling fee: endpoint $10,389.605193552, profit $389.605193552, and holding-period net return approximately 3.896052%.

Both model accounts are identical. Keep gross 4% and net account results separate. There is no portfolio increment this period, and no calendar holding duration to justify annualizing it.

Financial calculations use exact integer units of $1e−12 internally; display rounding does not rewrite the ledger. The recorded author check also used independent rational arithmetic. Agreement follows from calculations, not two AI opinions. Real C9 still lacks historical identities, membership, corporate actions, delisting outcomes, and execution evidence. A real same-pool benchmark, multiple-period risk, and capacity also need reconstruction. This is mechanism comparison, not full factor-project acceptance.

Repeat the exercise without writing code

Save the single offline HTML aid locally and open it in a browser. It requires no account, installation, or network. A document reader showing source is not a running tool: save the original file, then open that file with a browser. Website download integration is a separate, unverified path.

The current aid uses Chinese interface labels. Its visible steps 01, 02, and 03 locate the actions described here. In step 01, choose the original scenario—the first option—and use its training-and-prediction button. Copy both prediction columns and selected IDs before revealing labels. Write whether you expect smaller error necessarily to earn more.

Use step 02's reveal action, then compare each fee and cash amount in step 03. Export the CSV using step 03's export button. Check version, scenario, training/later inputs, both predictions, selections, errors, and endpoint. The input file itself contains all labels: hiding them in the display is reading order, not secure separation or a genuinely blind test.

Only after exporting the original choose the controlled variant, the second scenario option. Switching clears the display; training inputs and parameters stay fixed while later D's label changes. This is designed sensitivity, not a newly discovered market sample. Refresh does not preserve history: save both CSVs rather than overwrite the no-increment result.

The recorded author browser check exercised original and variant runs, label reveal, CSV downloads, script errors, external requests, and a 390-pixel layout in an isolated headless browser. It supports those specific tested paths. It is not a target reader's unaided test on a new computer or evidence that an application supports real-stock machine learning.

Compare maintenance as well as error

For real work, freeze question, asset pool, prediction start/end, training range, costs, and trial budget before examining later error, selection stability, and net portfolio outcomes. Smaller error may concern assets never selected. Or selection changes may create turnover whose costs exceed gross gains.

The example makes the transmission visible: OLS chooses D, the tree chooses C through its tie rule, and both assets gain 4%. The list changes while cash does not. Investigate whether an error improvement crosses both the selection and execution steps instead of stopping at an attractive error column.

Deep learning first needs a suitable task, adequate materials, a reliable simple baseline, and a manageable validation and maintenance budget. Reinforcement learning additionally needs credible actions, feedback, and an execution environment. Reward definitions or simulator loopholes can produce attractive training results. A model's name does not justify its use, and an LLM explanation cannot validate the environment.

Choose hardware from the measured bottleneck. This four-row task ran on a CPU path, with browser timings recorded for fitting and calculation. That is neither a long-term benchmark nor evidence of large-universe speed. Before considering GPU resources, measure model-specific data size, time, memory, repeated runs, and total cost, then compare hardware or cloud options. Hardware cannot repair missing labels, leakage, or absent economic explanation.

Change later D while keeping the model frozen

Explain why zero tree MAE does not establish better investing. For transfer, keep training, parameters, later x, tie rule, and costs fixed. Change only later D's gross label to 2%, so its D2 sale price is $102; C still sells at $104. Predict whether model outputs and selected IDs change, then whether the accounts remain equal. Finally select the controlled variant to check.

Predictions and IDs are unchanged: OLS still D, tree still C. Tree MAE becomes 0.5 percentage points and OLS 1.3 points; MSE becomes 0.0001 and 0.00022, respectively. OLS's sale notional is now $10,194.902448, selling commission $5.097451224, and endpoint $10,189.805095576, approximately 1.898051% net. The tree retains $10,389.605193552.

This variant creates a portfolio difference because a specified later input changed, not because training improved or independent evidence appeared. Increasing tree weight, changing depth, or retaining only this favorable variant starts another development attempt and must be logged. It cannot remain the original frozen comparison. The explanation question requires both points: prediction and selection objectives differ, and one constructed sample cannot establish future advantage.

Save T19 rather than only a winner's name

Keep version, task, sole feature, label availability, training/later ranges, two candidates and attempt budget, predictions/IDs, error units, fee-inclusive ledger, both scenario CSVs, resource measurements, and unverified items. Ask another reader to repeat key operations from the HTML and compare exports. If fields are absent or results disagree, record the blocker and first divergence. An author rerun cannot sign off the reader's attempt.

Compare the two predictions for the same asset, then follow selected IDs into prices and fees. The original should show unequal error but equal endpoints. The variant should show unchanged predictions and a lower OLS endpoint. Explaining both identifies what this T19 supports; retain the original, and leave real-strategy funding to separate evidence.

Trace a proposed stronger model through prediction, ranking, orders, and cash under the same conditions. If economic increment and maintenance requirements are unestablished, retaining a simple method or continuing research is reasonable. Upgrade for checkable added value, not an impressive tool name.

References

1. scikit-learn: Common pitfalls and recommended practices. Training/test separation and preprocessing leakage.
2. scikit-learn: Linear Models. OLS form and squared-error fitting. The fixed aid implements its own example; it does not call or claim to test this library.
3. scikit-learn: Decision Trees. Regression trees, piecewise-constant predictions, constraints, and limitations.

Sources checked October 30, 2026. The aid is offline and synthetic, without trading or data-service connections. AI-assisted draft; independent AI full-text and bilingual review is complete. Human financial review and target-reader testing have not been conducted and are not scheduled for this production round. Real C9 research remains incomplete.