Automatic differentation (AD) in Manopt requires the following: Matlab version R2021a or later. Deep Learning Toolbox version 14.2 or later. First, read the documentation of manoptAD by typing: help manoptAD Examples using AD can be found in the folder: /manopt/autodiff/basic_examples_AD More sophisticated examples can also be found in /examples by reading the comments inside the scripts. The comments here provide further information about how to use AD. These comments are necessary because of certain limitations both of the AD capabilities of the Deep Learning Toolbox and of Manopt's ability to handle certain delicate geometries, e.g., fixed-rank matrices, with AD. The basic usage of AD in Manopt goes as follows: problem.M = ...; %call a factory to get a manifold structure problem.cost = @(x) ...; % implement your cost function problem = manoptAD(problem); % ask AD to figure out gradient + Hessian ...; % call a solver on problem, e.g., x = trustregions(problem); The current implementation of AD is based on Matlab's Deep Learning Toolbox. The latter relies on dlarray to represent data (arrays). While this works very well, unfortunately, there are a few limitations. For example, certain functions do not support dlarray. See the following official website for a list of compatbile functions: https://ch.mathworks.com/help/deeplearning/ug/list-of-functions-with-dlarray-support.html. Moreover, dlarray only supports complex variables starting with Matlab R2021b. To handle complex numbers in R2021a and earlier, and also to handle functions that are not supported by dlarray at this moment, Manopt provides a limited collection of backup functions which are compatible with AD. Explicitly, the left column below lists commonly used functions which are not supported by dlarray at this time, and the right column lists corresponding replacement functions that can be used: trace ctrace diag cdiag triu ctriu tril ctril norm(..., 'fro') cnormfro norm(..., 'fro')^2 cnormsqfro multiscale cmultiscale All the other multi*** functions in Manopt support AD. Moreover, bsxfun is not supported. The user may have to translate it into repmat and point-wise expressions. Concatenating arrays along the third or higher dimension such as in cat(A, B, 3+) is not supported for AD. The matrix functions sqrtm, logm, expm, eig, svd, det, cumsum, movsum, cumprod, \, inv, mod, rem, vecnorm, bandwidth are not supported. For Matlab R2021a or earlier, in addition to the functions mentioned above, dot is not supported. Element-wise multiplication can be replaced by cdottimes. To deal with complex variables in R2021a and earlier, Manopt converts complex arrays into a structure with fields real and imag. See mat2dl_complex and dl2mat_complex for more details. In this case, the user should try using the basic functions in the folder /manopt/autodiff/functions_AD when defining the cost function. An alternative way is to define one's own basic functions. These functions should accept both numeric arrays and structures with fields real and imag. See cprod and complex_example_AD as examples. See also: manoptAD complex_example_AD
0001 function manoptADhelp() 0002 % Automatic differentation (AD) in Manopt requires the following: 0003 % Matlab version R2021a or later. 0004 % Deep Learning Toolbox version 14.2 or later. 0005 % 0006 % First, read the documentation of manoptAD by typing: 0007 % 0008 % help manoptAD 0009 % 0010 % Examples using AD can be found in the folder: 0011 % 0012 % /manopt/autodiff/basic_examples_AD 0013 % 0014 % More sophisticated examples can also be found in 0015 % 0016 % /examples 0017 % 0018 % by reading the comments inside the scripts. 0019 % 0020 % The comments here provide further information about how to use AD. These 0021 % comments are necessary because of certain limitations both of the AD 0022 % capabilities of the Deep Learning Toolbox and of Manopt's ability to 0023 % handle certain delicate geometries, e.g., fixed-rank matrices, with AD. 0024 % 0025 % The basic usage of AD in Manopt goes as follows: 0026 % 0027 % problem.M = ...; %call a factory to get a manifold structure 0028 % problem.cost = @(x) ...; % implement your cost function 0029 % problem = manoptAD(problem); % ask AD to figure out gradient + Hessian 0030 % ...; % call a solver on problem, e.g., x = trustregions(problem); 0031 % 0032 % The current implementation of AD is based on Matlab's Deep Learning 0033 % Toolbox. The latter relies on dlarray to represent data (arrays). 0034 % While this works very well, unfortunately, there are a few limitations. 0035 % For example, certain functions do not support dlarray. 0036 % See the following official website for a list of compatbile functions: 0037 % 0038 % https://ch.mathworks.com/help/deeplearning/ug/list-of-functions-with-dlarray-support.html. 0039 % 0040 % Moreover, dlarray only supports complex variables starting with 0041 % Matlab R2021b. 0042 % 0043 % To handle complex numbers in R2021a and earlier, and also to handle 0044 % functions that are not supported by dlarray at this moment, Manopt 0045 % provides a limited collection of backup functions which are compatible 0046 % with AD. Explicitly, the left column below lists commonly used functions 0047 % which are not supported by dlarray at this time, and the right column 0048 % lists corresponding replacement functions that can be used: 0049 % 0050 % trace ctrace 0051 % diag cdiag 0052 % triu ctriu 0053 % tril ctril 0054 % norm(..., 'fro') cnormfro 0055 % norm(..., 'fro')^2 cnormsqfro 0056 % multiscale cmultiscale 0057 % 0058 % All the other multi*** functions in Manopt support AD. 0059 % 0060 % Moreover, bsxfun is not supported. The user may have to translate it 0061 % into repmat and point-wise expressions. 0062 % Concatenating arrays along the third or higher dimension such as in 0063 % cat(A, B, 3+) is not supported for AD. 0064 % The matrix functions sqrtm, logm, expm, eig, svd, det, cumsum, movsum, 0065 % cumprod, \, inv, mod, rem, vecnorm, bandwidth are not supported. 0066 % 0067 % For Matlab R2021a or earlier, in addition to the functions mentioned 0068 % above, dot is not supported. 0069 % Element-wise multiplication can be replaced by cdottimes. 0070 % 0071 % To deal with complex variables in R2021a and earlier, Manopt converts 0072 % complex arrays into a structure with fields real and imag. 0073 % See mat2dl_complex and dl2mat_complex for more details. In this case, 0074 % the user should try using the basic functions in the folder 0075 % 0076 % /manopt/autodiff/functions_AD 0077 % 0078 % when defining the cost function. An alternative way is to define one's 0079 % own basic functions. These functions should accept both numeric arrays 0080 % and structures with fields real and imag. 0081 % See cprod and complex_example_AD as examples. 0082 % 0083 % See also: manoptAD complex_example_AD 0084 0085 % This file is part of Manopt: www.manopt.org. 0086 % Original author: Xiaowen Jiang, Aug. 31, 2021. 0087 % Contributors: Nicolas Boumal 0088 % Change log: 0089 0090 fprintf('The file manoptADhelp is just for documentation: run ''help manoptADhelp''.\n'); 0091 0092 end