Some function spaces and Universal Aproximation Theorem

The following Theorems are results obtained from reference 1,

Spaces of Continuos Funtions

Theorem

If $\sigma$ is continuos, bounded and nonconstant, then $\mathcal{N}_n(\sigma)$ is dense in $C(\Omega)$ for all compact subsets $\Omega$ of $\mathbb{R}^n$

Lebesgue Spaces

Theorem

If $\sigma$ is bounded and nonconstant, then $\mathcal{N}_n(\sigma)$ is dense in $L^p(\mu)$ for all finites measures $\mu$ on $\mathbb{R}^n$

Multilayer

Multilayer feedforward neural networks are also universal approximators of functions, extending the well-known results for single hidden layer networks to deeper architectures. This generalization is formally established in Corollary 2.6 from reference 2, which shows that the same approximation capabilities apply to networks with multiple hidden layers, as long as the activation functions are suitably chosen and the number of hidden units is sufficient.

Multioutput

Similarly, the universal approximation results extend not only to multilayer networks but also to networks with multiple outputs. This generalization is also established in Corollary 2.6 of reference of 2.

References

1. K. Hornik, Approximation capabilities of multilayer feedforward networks, Neural Networks, vol. 4, no. 2, pp. 251–257, 1991.

2. K. Hornik, M. Stinchcombe, and H. White, Multilayer feedforward networks are universal approximators, Neural Networks, vol. 2, no. 5, pp. 359–366, 1989.