Nested Clustered Optimization (NCO)

Overview

Nested Clustered Optimization (NCO), introduced by Marcos Lopez de Prado and Michael J. Lewis in 2019, represents an innovative portfolio construction framework that combines the advantages of hierarchical clustering with traditional mean-variance optimization techniques.

NCO bridges the gap between purely algorithmic approaches like Hierarchical Risk Parity (HRP) and traditional optimization methods. It leverages the structure discovered through clustering while allowing for the incorporation of expected returns and various objective functions, making it a powerful and flexible approach for modern portfolio management.

Intuitive Explanation

Think of NCO as a structured team decision-making process:

Imagine you're managing a large corporation with many divisions. Instead of making all decisions at the top level, you first organize similar divisions into departments (clustering). Then, within each department, you optimize resources based on specific goals. Finally, you allocate budget across departments based on their relative importance to the company.

In NCO:

• Assets are first grouped into clusters based on their similarities (like HRP).

• Within each cluster, a traditional optimization method (such as mean-variance) is applied.

• These optimized clusters are then treated as "meta-assets" for a final portfolio-wide optimization.

This multi-level approach combines the stability of clustering with the efficiency of optimization techniques, offering a balance between structure and flexibility.

Detailed Mathematical & Algorithmic Explanation

NCO follows a systematic approach that can be broken down into three primary stages:

Step 1: Hierarchical Clustering

Similar to HRP, NCO begins by grouping assets based on their correlations using a distance measure:

where:

• $d_{i,j}$ is the distance between assets $i$ and $j$.
• $\rho_{i,j}$ is the correlation between asset returns.

This clustering creates a hierarchical structure (dendrogram) and determines which assets belong to which clusters.

Step 2: Intra-Cluster Optimization

For each identified cluster, a separate optimization problem is solved:

where:

• $w_c^*$ is the optimal weight vector for cluster $c$.
• $f$ is an objective function (e.g., mean-variance utility, Sharpe ratio).
• $\mu_c$ is the expected return vector for assets in cluster $c$.
• $\Sigma_c$ is the covariance matrix for assets in cluster $c$.

This step can use any of the traditional optimization approaches such as:

• Maximum Sharpe ratio: Maximizing $\frac{\mu^T w - r_f}{\sqrt{w^T \Sigma w}}$

• Minimum variance: Minimizing $w^T \Sigma w$

• Mean-variance utility: Maximizing $\mu^T w - \lambda w^T \Sigma w$

Step 3: Inter-Cluster Allocation

Each optimized cluster is now treated as a single "meta-asset" with:

• Expected return: $\mu_c^* = w_c^{*T} \mu_c$
• Variance: $\sigma_c^{*2} = w_c^{*T} \Sigma_c w_c^*$

The final portfolio optimization is performed across these meta-assets:

where:

• $v^*$ is the optimal weight vector for the meta-assets.
• $\mu^*$ is the vector of expected returns for meta-assets.
• $\Sigma^*$ is the covariance matrix of meta-assets.

The final portfolio weights are computed by combining the intra-cluster weights with the inter-cluster allocation:

where:

• $w_i$ is the final weight of asset $i$ in the portfolio.
• $v_c^*$ is the optimal weight of cluster $c$ in the portfolio.
• $w_{i,c}^*$ is the optimal weight of asset $i$ within cluster $c$.

Objective Functions

NCO allows for various objective functions at both intra-cluster and inter-cluster levels, including:

Advantages and Limitations

References

• Lopez de Prado, M., & Lewis, M. J. (2019). "Nested Clustered Optimization: A Clustering-Based Portfolio Construction Algorithm." SSRN Electronic Journal. doi:10.2139/ssrn.3469961

• Lopez de Prado, M. (2020). "Machine Learning for Asset Managers," Cambridge University Press. doi:10.1017/9781108883658

• Sjöstrand, D., & Behnejad, N. (2020). "Exploration of hierarchical clustering in long-only risk-based portfolio optimization." Master's thesis, Copenhagen Business School.

• Riskfolio-Lib Documentation: Hierarchical Clustering Portfolio Optimization