Automatic differentation (AD) in Manopt requires the following:
Matlab version R2021a or later.
Deep Learning Toolbox version 14.2 or later.
First, read the documentation of manoptAD by typing:
help manoptAD
Examples using AD can be found in the folder:
/manopt/autodiff/basic_examples_AD
More sophisticated examples can also be found in
/examples
by reading the comments inside the scripts.
The comments here provide further information about how to use AD. These
comments are necessary because of certain limitations both of the AD
capabilities of the Deep Learning Toolbox and of Manopt's ability to
handle certain delicate geometries, e.g., fixed-rank matrices, with AD.
The basic usage of AD in Manopt goes as follows:
problem.M = ...; %call a factory to get a manifold structure
problem.cost = @(x) ...; % implement your cost function
problem = manoptAD(problem); % ask AD to figure out gradient + Hessian
...; % call a solver on problem, e.g., x = trustregions(problem);
The current implementation of AD is based on Matlab's Deep Learning
Toolbox. The latter relies on dlarray to represent data (arrays).
While this works very well, unfortunately, there are a few limitations.
For example, certain functions do not support dlarray.
See the following official website for a list of compatbile functions:
https://ch.mathworks.com/help/deeplearning/ug/list-of-functions-with-dlarray-support.html.
Moreover, dlarray only supports complex variables starting with
Matlab R2021b.
To handle complex numbers in R2021a and earlier, and also to handle
functions that are not supported by dlarray at this moment, Manopt
provides a limited collection of backup functions which are compatible
with AD. Explicitly, the left column below lists commonly used functions
which are not supported by dlarray at this time, and the right column
lists corresponding replacement functions that can be used:
trace ctrace
diag cdiag
triu ctriu
tril ctril
norm(..., 'fro') cnormfro
norm(..., 'fro')^2 cnormsqfro
multiscale cmultiscale
All the other multi*** functions in Manopt support AD.
Moreover, bsxfun is not supported. The user may have to translate it
into repmat and point-wise expressions.
Concatenating arrays along the third or higher dimension such as in
cat(A, B, 3+) is not supported for AD.
The matrix functions sqrtm, logm, expm, eig, svd, det, cumsum, movsum,
cumprod, \, inv, mod, rem, vecnorm, bandwidth are not supported.
For Matlab R2021a or earlier, in addition to the functions mentioned
above, dot is not supported.
Element-wise multiplication can be replaced by cdottimes.
To deal with complex variables in R2021a and earlier, Manopt converts
complex arrays into a structure with fields real and imag.
See mat2dl_complex and dl2mat_complex for more details. In this case,
the user should try using the basic functions in the folder
/manopt/autodiff/functions_AD
when defining the cost function. An alternative way is to define one's
own basic functions. These functions should accept both numeric arrays
and structures with fields real and imag.
See cprod and complex_example_AD as examples.
See also: manoptAD complex_example_AD0001 function manoptADhelp() 0002 % Automatic differentation (AD) in Manopt requires the following: 0003 % Matlab version R2021a or later. 0004 % Deep Learning Toolbox version 14.2 or later. 0005 % 0006 % First, read the documentation of manoptAD by typing: 0007 % 0008 % help manoptAD 0009 % 0010 % Examples using AD can be found in the folder: 0011 % 0012 % /manopt/autodiff/basic_examples_AD 0013 % 0014 % More sophisticated examples can also be found in 0015 % 0016 % /examples 0017 % 0018 % by reading the comments inside the scripts. 0019 % 0020 % The comments here provide further information about how to use AD. These 0021 % comments are necessary because of certain limitations both of the AD 0022 % capabilities of the Deep Learning Toolbox and of Manopt's ability to 0023 % handle certain delicate geometries, e.g., fixed-rank matrices, with AD. 0024 % 0025 % The basic usage of AD in Manopt goes as follows: 0026 % 0027 % problem.M = ...; %call a factory to get a manifold structure 0028 % problem.cost = @(x) ...; % implement your cost function 0029 % problem = manoptAD(problem); % ask AD to figure out gradient + Hessian 0030 % ...; % call a solver on problem, e.g., x = trustregions(problem); 0031 % 0032 % The current implementation of AD is based on Matlab's Deep Learning 0033 % Toolbox. The latter relies on dlarray to represent data (arrays). 0034 % While this works very well, unfortunately, there are a few limitations. 0035 % For example, certain functions do not support dlarray. 0036 % See the following official website for a list of compatbile functions: 0037 % 0038 % https://ch.mathworks.com/help/deeplearning/ug/list-of-functions-with-dlarray-support.html. 0039 % 0040 % Moreover, dlarray only supports complex variables starting with 0041 % Matlab R2021b. 0042 % 0043 % To handle complex numbers in R2021a and earlier, and also to handle 0044 % functions that are not supported by dlarray at this moment, Manopt 0045 % provides a limited collection of backup functions which are compatible 0046 % with AD. Explicitly, the left column below lists commonly used functions 0047 % which are not supported by dlarray at this time, and the right column 0048 % lists corresponding replacement functions that can be used: 0049 % 0050 % trace ctrace 0051 % diag cdiag 0052 % triu ctriu 0053 % tril ctril 0054 % norm(..., 'fro') cnormfro 0055 % norm(..., 'fro')^2 cnormsqfro 0056 % multiscale cmultiscale 0057 % 0058 % All the other multi*** functions in Manopt support AD. 0059 % 0060 % Moreover, bsxfun is not supported. The user may have to translate it 0061 % into repmat and point-wise expressions. 0062 % Concatenating arrays along the third or higher dimension such as in 0063 % cat(A, B, 3+) is not supported for AD. 0064 % The matrix functions sqrtm, logm, expm, eig, svd, det, cumsum, movsum, 0065 % cumprod, \, inv, mod, rem, vecnorm, bandwidth are not supported. 0066 % 0067 % For Matlab R2021a or earlier, in addition to the functions mentioned 0068 % above, dot is not supported. 0069 % Element-wise multiplication can be replaced by cdottimes. 0070 % 0071 % To deal with complex variables in R2021a and earlier, Manopt converts 0072 % complex arrays into a structure with fields real and imag. 0073 % See mat2dl_complex and dl2mat_complex for more details. In this case, 0074 % the user should try using the basic functions in the folder 0075 % 0076 % /manopt/autodiff/functions_AD 0077 % 0078 % when defining the cost function. An alternative way is to define one's 0079 % own basic functions. These functions should accept both numeric arrays 0080 % and structures with fields real and imag. 0081 % See cprod and complex_example_AD as examples. 0082 % 0083 % See also: manoptAD complex_example_AD 0084 0085 % This file is part of Manopt: www.manopt.org. 0086 % Original author: Xiaowen Jiang, Aug. 31, 2021. 0087 % Contributors: Nicolas Boumal 0088 % Change log: 0089 0090 fprintf('The file manoptADhelp is just for documentation: run ''help manoptADhelp''.\n'); 0091 0092 end