antsRegistration does not produce equivalent results #210

dmirman-zz · 2018-03-29T14:23:38Z

Multiple runs of antsRegistration() on the same images produces different results. Here is a simple example:

library(ANTsR)

template <- antsImageRead(getANTsRData("ch2")) #colins template
atlas <- antsImageRead(getANTsRData("mnia")) #aal atlas

#register template space to atlas
regRigid <- list()
regRigidcount <- NA
for(i in 1:10){
  regRigid[[i]] <- antsRegistration(fixed = atlas, moving = template, typeofTransform = c("Rigid"))
  regRigidcount[[i]] <- sum(regRigid[[i]]$warpedmovout)
  print(regRigidcount[[i]])
}
# the counts are not the same for the 10 runs of the registration

Similarly non-identical results show up for other transforms ("Affine", "SyNCC"). Although the differences seem somewhat small, it is not clear why they are different at all. More importantly, these small differences have substantive consequences down-stream: we ran a lesion-symptom mapping analysis and wanted to describe the results in terms of the AAL atlas, but the 10 somewhat different transformations produce somewhat different descriptions:

#read in data to cluster
# file is available from http://drive.google.com/file/d/1tm2PI6yxhEngB60jNjoP6YQ3T_H7u8JM"
data <- antsImageRead("LSM_result.nii") 

#Rigid Transformation
wrpRigid <- list()
clustRigid <- list()
statRigid <- list()
for(a in 1:length(regRigid)){
  wrpRigid[[a]] <- antsApplyTransforms(fixed = atlas, moving = data, 
                                      transformlist = regRigid[[a]]$fwdtransforms, interpolator = c("NearestNeighbor"))
  clustRigid[[a]] <- labelClusters(wrpRigid[[a]], maxThresh = 1, minClusterSize = 0, minThresh = 0.1)
  statRigid[[a]] <- labelStats(wrpRigid[[a]], clustRigid[[a]])
  print(statRigid[[a]])
}

In my test, I get anywhere from 7 to 11 clusters, with slightly different masses and (x,y,z) coordinates.

This was run most recently using R version 3.3.1, ANTsR version: 0.5.6, ANTsRCore version 0.3.4, but we've replicated this problem on two different Linux machines.

The text was updated successfully, but these errors were encountered:

ntustison · 2018-03-29T15:03:59Z

Please see this thread where these issues are discussed.

stnava · 2018-03-29T15:15:21Z

just a comment on the scientific result - this would suggest ( already fairly clear from the literature ) that univariate VLSM results are not very stable. of course, one can exploit this uncertainty in order to better understand the nature of the estimation problem. worth adding some of this information to the `antsRegistration` documentation in order to head off repeat discussion. will leave this open until a contribution to the DOC is made by myself or someone else. brian

…

On Thu, Mar 29, 2018 at 11:04 AM, Nick Tustison ***@***.***> wrote: Please see this thread <ANTsX/ANTs#444> where these issues are discussed. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#210 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AATyftSHh8knfPsPh2r_bEfQWQ43J8SIks5tjPffgaJpZM4TAbCv> .

muschellij2 · 2018-03-29T15:43:27Z

It seems like that thread (ANTsX/ANTs#444 (comment)) indicated changing --float to --double may help, is that correct?

If so, what about changing
https://github.com/ANTsX/ANTsRCore/blob/master/R/antsRegistration.R#L494
to --double.

dmirman-zz · 2018-03-29T15:50:12Z

That was also my understanding of that thread, but it doesn't seem like antsRegistration will accept a --double flag. When I try
regRigid[[i]] <- antsRegistration(fixed = atlas, moving = template, typeofTransform = c("Rigid"), "--double")
I get
ERROR: Invalid flag provided double

stnava · 2018-03-29T15:52:34Z

Massive impact on memory use and creates more maintenance overhead Due to need for converting between types ... furthermore, no guarantee it will fix the issue. Running in single threaded mode without random sampling in the metric is probably good enough. Although I have not tested that recently.

On Thu, Mar 29, 2018 at 11:43 AM John Muschelli ***@***.***> wrote: It seems like that thread (ANTsX/ANTs#444 (comment) <ANTsX/ANTs#444 (comment)>) indicated changing --float to --double may help, is that correct? If so, what about changing https://github.com/ANTsX/ANTsRCore/blob/master/R/antsRegistration.R#L494 to --double. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#210 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AATyfn2ztlIqa_SYDjAyksVVLwzrZBH2ks5tjQEngaJpZM4TAbCv> .

-- brian

dmirman-zz · 2018-03-29T16:10:17Z

I'd like to test running this in single threaded mode without random sampling, but how do I specify that within ANTsR?

cookpa · 2018-03-29T16:47:42Z

I think you would need dense sampling (strategy "NONE") to have completely reproducible results. With regular sampling, a perturbation is applied to the points to randomize their location within the voxel grid.

https://github.com/InsightSoftwareConsortium/ITK/blob/8a2a15f41218c925c0a89119e09419d48f83eb22/Modules/Registration/RegistrationMethodsv4/include/itkImageRegistrationMethodv4.hxx#L954-L984

dmirman-zz · 2018-03-29T21:13:41Z

Please pardon my ignorance, but I do not know enough about how ANTsR communicates with ITK (or about how ITK works) in order to implement the solutions that have been described here. As I understand it, two strategies have been proposed:
(1) use --double precision, but antsRegistration doesn't seem to take that as a flag and there may be overly negative consequences for memory use.
(2) run in single threaded mode without random sampling and use dense sampling. It's not clear to me how to control those parameters from within ANTsR.

Any advice about implementing either or both of these would be most appreciated.

ntustison · 2018-03-29T21:39:06Z

antsRegistration, as defined in ANTsR, is used more like a script which simplifies the underlying antsRegistration tool in ANTs. If you need to perform your experiments within ANTsR, you might want to access the underlying program using the antsRegistration( list_of_arguments ) format. For example, to see the short and long help menus, you can type

> antsRegistration( list( '-h', 1 ) )
> antsRegistration( list( '--help', 1 ) )

You could then use double by specifying "--float 0", i.e.,

> args <- list( "-d", ... "--float", "0", ... )
> antsRegistration( args )

See here for an example. In the same file you can see how all the different transforms are defined.

dorianps · 2018-03-30T01:28:09Z

I find this issue quite disturbing. Has anybody tried to check all three options to see which one resolves the problem, or whether they are all valid approaches to enhance reproducibility in an additive manner: - double/float - sample density (default 0.25) - single core vs multiple core Probably this should be part of a larger effort to enhance registration reproducibility. Dorian

…

On Thu, Mar 29, 2018, 17:39 Nick Tustison ***@***.***> wrote: antsRegistration, as defined in ANTsR, is used more like a script which simplifies the underlying antsRegistration tool in ANTs. If you need to perform your experiments within ANTsR, you might want to access the underlying program using the antsRegistration( list_of_arguments ) format. For example, to see the short and long help menus, you can type > antsRegistration( list( '-h', 1 ) ) > antsRegistration( list( '--help', 1 ) ) You could then use double by specifying "--float 0", i.e., > args <- list( "-d", ... "--float", "0", ... ) > antsRegistration( args ) See here <https://github.com/ANTsX/ANTsRCore/blob/master/R/antsRegistration.R#L284-L294> for an example. In the same file you can see how all the different transforms are defined. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#210 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AIqafd5asjfB59sBWQheJqLoP-BNvKFFks5tjVR7gaJpZM4TAbCv> .

stnava · 2018-03-30T12:07:09Z

the solution: Go into Itk ; set a fixed seed; no multi threading... should be very reproducible. But you lose multithreading speed-up and hide the uncertainty intrinsic to the problem. Choose your poison. If the population statistical outputs are sensitive to tiny variations in rotation parameters, then there are other issues one should worry about. There were a few papers that worked on voting schemes for registration. The point was to acknowledge that the solutions are uncertain and then to use that to bootstrap the estimate ie find a median solution or something along those lines. anyway, not much here to justify new work. if you want to open an issue at ITK, then point to this: https://github.com/InsightSoftwareConsortium/ITK/blob/ 8a2a15f41218c925c0a89119e09419d48f83eb22/Modules/Registration/ RegistrationMethodsv4/include/itkImageRegistrationMethodv4.hxx#L940-L949 which you would want to set as a fixed seed. however, there is a cost to that - how do you choose the seed? then, you might ask - why perturb the points at all? it's b/c there is estimation bias if you sample the grid in a perfectly regular manner. http://bigwww.epfl.ch/preprints/thevenaz0602p.pdf and, again, there is the fundamental computational issue: https://en.wikipedia.org/wiki/Kahan_summation_algorithm frequent issue in scientific computing.

ntustison · 2018-03-30T13:38:25Z

Thanks Brian--informative as always. I added this wiki page as an initial attempt for addressing the relevant issues. Let me know if you'd like something different.

dmirman-zz · 2018-04-02T14:10:47Z

I can confirm that using double precision is computationally impractical: I tried it for a Rigid registration and it still hadn't finished after several hours (float version takes ~20sec). Thanks everyone for an informative discussion.

stnava · 2018-04-02T14:32:33Z

should not lead to that much computation time difference - unless you are running out of memory. brian

…

On Mon, Apr 2, 2018 at 10:10 AM, Dan Mirman ***@***.***> wrote: I can confirm that using double precision is computationally impractical: I tried it for a Rigid registration and it still hadn't finished after several hours (float version takes ~20sec). Thanks everyone for an informative discussion. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#210 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AATyfgZOXHsFWPU37Ks4UzuKEMwH9PBlks5tkjFngaJpZM4TAbCv> .

stnava · 2018-06-03T15:59:35Z

we can close this issue due to several recent changes in ANTs and ANTsR. you will need to define (in your environment) ANTS_RANDOM_SEED=1 (or whatever your favorite number is). e.g. my .Renviron file reads:

ANTS_RANDOM_SEED=1234
ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS=1

this should give you repeatable results.

ultimately, we may also allow a command line parameter to control the seed.

one note: histogram matching induced an additional unexpected source of random variability.

@ncullen93 may find this of interest as well.

moskvich412 · 2019-09-27T21:32:09Z

In testing ANTs registration, we discovered it is not reproducible: running several times on the same inputs gives different morphisms. Implementing:

ANTS_RANDOM_SEED=1234
ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS=1

as suggested above, does not seem to completely resolve the issue. I see based on CPU usage that threads are not created. But it is impossible for me to check if fixing seed took effect. We are using version 2.3.1, should these variables have effect in this version?

Are there other methods to enforce reproducibility?

P.S. It is not obvious to me how randomness can help solving for brain morphism (random seed suggests it is done on purpose), but this is not the question at hand.

ntustison · 2019-09-28T15:15:47Z

@moskvich412 I see that you have read a couple of the comments in this thread, specifically the last comment from @stnava dealing with ANTS_RANDOM_SEED and ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS but have you read some of the other comments pointing to some of the foundational problems with this issue that goes beyond ANTs? There are even some references pointing to discussion of these algorithmic stochastic elements and why they're important, including in the study of brain morphisms.

moskvich412 · 2019-09-28T15:36:08Z

Yes, i did read some references, some paper on some sampling strategy, and why one by some name i can not remember which is generalization to 3D of some other name that i can not remember than uniform sampling. Other things on importance of order of summation and threads. However, my reading does not answer the question how to make ANTs reproducible. If humans are asked to segment same brain several times, the results will be different. Why? Because in some tough situations humans will not know how to decide, and not remember what they did last time. If ANTs wants to mimic that aspect of human behavior, then it is likely successful. I consider this a limitation of human brain and would like ANTs to surpass humans. Why? Because when we reprocess data and get different result it is very hard to understand the cause when things are intrinsically unstable. We have to study the effect of morphism (or anything else) variability on the results, and try to understand if the observed deviation of a given reprocessing is outside of the natural variability. Putting philosophy aside, did i do the right thing to make ANTs reproducible for the version we have? Is there else to try? Is remaining variability intrinsic and can not be turned off? Is there another version of ANTs which has the same morphism quality and reproducible? Roman Sent from my T-Mobile 4G LTE Device

…

-------- Original message -------- From: Nick Tustison <notifications@github.com> Date: 9/28/19 11:15 AM (GMT-05:00) To: ANTsX/ANTsR <ANTsR@noreply.github.com> Cc: Roman Fleysher <roman.fleysher@einstein.yu.edu>, Mention <mention@noreply.github.com> Subject: Re: [ANTsX/ANTsR] antsRegistration does not produce equivalent results (#210) @moskvich412<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmoskvich412&data=02%7C01%7Croman.fleysher%40einstein.yu.edu%7C33de4fc91da64ea5411e08d74426bfbd%7C04c70eb48f2648079934e02e89266ad0%7C1%7C0%7C637052805523853370&sdata=PPIjQ64e3g2GghxMguW8aHKCy6pimGKUxVMYPo8Vvlk%3D&reserved=0> I see that you have read a couple of the comments in this thread, specifically the last comment from @stnava<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fstnava&data=02%7C01%7Croman.fleysher%40einstein.yu.edu%7C33de4fc91da64ea5411e08d74426bfbd%7C04c70eb48f2648079934e02e89266ad0%7C1%7C0%7C637052805523863368&sdata=OYxzyrErwHngYrHCXNqKpqk4SCoVYRvJ%2BEaTQSVMef4%3D&reserved=0> dealing with ANTS_RANDOM_SEED and ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS but have you read some of the other comments pointing to some of the foundational problems with this issue that goes beyond ANTs? There are even some references pointing to discussion of these algorithmic stochastic elements and why they're important, including in the study of brain morphisms. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FANTsX%2FANTsR%2Fissues%2F210%3Femail_source%3Dnotifications%26email_token%3DANKPD2Z3HRSXZPY22UWIDIDQL5YKNA5CNFSM4EYBWCX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD723Y4Y%23issuecomment-536198259&data=02%7C01%7Croman.fleysher%40einstein.yu.edu%7C33de4fc91da64ea5411e08d74426bfbd%7C04c70eb48f2648079934e02e89266ad0%7C1%7C0%7C637052805523863368&sdata=4CTnxTRrSqwerGEcuhIDU9q5JPmPobduKBJszI9ekvA%3D&reserved=0>, or mute the thread<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FANKPD25DASAL3YHI3N2BMCDQL5YKNANCNFSM4EYBWCXQ&data=02%7C01%7Croman.fleysher%40einstein.yu.edu%7C33de4fc91da64ea5411e08d74426bfbd%7C04c70eb48f2648079934e02e89266ad0%7C1%7C0%7C637052805523863368&sdata=KMiHHbkN1CzV6CPxrtE0QSEM9h4dCnVA5NxP3a9FMBA%3D&reserved=0>.

ntustison · 2019-09-28T15:43:22Z

You've probably done all you can to remove ANTs variability.

cookpa · 2019-09-28T16:17:23Z

I thought fixing the seed took care of this, but if you can share a code and data I may be able to say more.

…

Sent from my iPhone

On Sep 28, 2019, at 11:36 AM, moskvich412 ***@***.***> wrote: Putting philosophy aside, did i do the right thing to make ANTs reproducible for the version we have? Is there else to try? Is remaining variability intrinsic and can not be turned off? Is there another version of ANTs which has the same morphism quality and reproducible?

moskvich412 · 2019-09-28T16:30:21Z

There is no code to share. I am running compiled code. I tried on mouse and human, several pairs, several runs on each pair. They look similar, but visibly different. I am not talking about 4th decimal place. Likely second decimal. I thought fixing the seed took care of this, but if you can share a code and data I may be able to say more.

Sent from my iPhone

On Sep 28, 2019, at 11:36 AM, moskvich412 ***@***.***> wrote: Putting philosophy aside, did i do the right thing to make ANTs reproducible for the version we have? Is there else to try? Is remaining variability intrinsic and can not be turned off? Is there another version of ANTs which has the same morphism quality and reproducible?

—

cookpa · 2019-09-28T16:36:13Z

I mean the commands that you ran

Sent from my iPhone On Sep 28, 2019, at 12:30 PM, moskvich412 <notifications@github.com<mailto:notifications@github.com>> wrote: There is no code to share. I am running compiled code. I tried on mouse and human, several pairs, several runs on each pair. They look similar, but visibly different. I am not talking about 4th decimal place. Likely second decimal.

-------- Original message -------- From: Philip Cook <notifications@github.com<mailto:notifications@github.com>> Date: 9/28/19 12:17 PM (GMT-05:00) To: ANTsX/ANTsR <ANTsR@noreply.github.com<mailto:ANTsR@noreply.github.com>> Cc: Roman Fleysher <roman.fleysher@einstein.yu.edu<mailto:roman.fleysher@einstein.yu.edu>>, Mention <mention@noreply.github.com<mailto:mention@noreply.github.com>> Subject: Re: [ANTsX/ANTsR] antsRegistration does not produce equivalent results (#210) I thought fixing the seed took care of this, but if you can share a code and data I may be able to say more.

Sent from my iPhone

On Sep 28, 2019, at 11:36 AM, moskvich412 ***@***.******@***.***>> wrote: Putting philosophy aside, did i do the right thing to make ANTs reproducible for the version we have? Is there else to try? Is remaining variability intrinsic and can not be turned off? Is there another version of ANTs which has the same morphism quality and reproducible?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FANTsX%2FANTsR%2Fissues%2F210%3Femail_source%3Dnotifications%26email_token%3DANKPD24CUWKORS2STSGQMHDQL57RLA5CNFSM4EYBWCX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD725CCY%23issuecomment-536203531&data=02%7C01%7Croman.fleysher%40einstein.yu.edu%7Ca1a4cc38f43a4440e2a708d7442f5991%7C04c70eb48f2648079934e02e89266ad0%7C1%7C0%7C637052842480395917&sdata=xqCQliG34AIBYQ%2FYTAB7q3I5RfgsQZGKLwGMSEcwbkE%3D&reserved=0>, or mute the thread<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FANKPD25IFXJSB6USGMZFYQ3QL57RLANCNFSM4EYBWCXQ&data=02%7C01%7Croman.fleysher%40einstein.yu.edu%7Ca1a4cc38f43a4440e2a708d7442f5991%7C04c70eb48f2648079934e02e89266ad0%7C1%7C0%7C637052842480405914&sdata=Ty%2FPv1JYtg35dSlVfAaMQsCq4WMC7RpQVBSrYmKuuCg%3D&reserved=0>. — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#210?email_source=notifications&email_token=AAEVJM2C4QUFM7L7NVMJGK3QL6BB5A5CNFSM4EYBWCX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD725KMI#issuecomment-536204593>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AAEVJM6GV4534DDBLFEFMKDQL6BB5ANCNFSM4EYBWCXQ>.

dorianps · 2019-09-28T17:16:47Z

I am also interested in having a reproducible=TRUE in antsRegistration(). Would be good to have ANTsR itself take care of the needed arguments (seed and threads) to achieve it, and perhaps return the seed number for future use.

…

On Sat, Sep 28, 2019, 12:36 PM Philip Cook ***@***.***> wrote: I mean the commands that you ran Sent from my iPhone On Sep 28, 2019, at 12:30 PM, moskvich412 ***@***.*** ***@***.***>> wrote: There is no code to share. I am running compiled code. I tried on mouse and human, several pairs, several runs on each pair. They look similar, but visibly different. I am not talking about 4th decimal place. Likely second decimal. -------- Original message -------- From: Philip Cook ***@***.***<mailto: ***@***.***>> Date: 9/28/19 12:17 PM (GMT-05:00) To: ANTsX/ANTsR ***@***.******@***.***>> Cc: Roman Fleysher ***@***.***<mailto: ***@***.***>>, Mention ***@***.*** ***@***.***>> Subject: Re: [ANTsX/ANTsR] antsRegistration does not produce equivalent results (#210) I thought fixing the seed took care of this, but if you can share a code and data I may be able to say more. Sent from my iPhone > On Sep 28, 2019, at 11:36 AM, moskvich412 ***@***.*** ***@***.***>> wrote: > > Putting philosophy aside, did i do the right thing to make ANTs reproducible for the version we have? Is there else to try? Is remaining variability intrinsic and can not be turned off? Is there another version of ANTs which has the same morphism quality and reproducible? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub< https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FANTsX%2FANTsR%2Fissues%2F210%3Femail_source%3Dnotifications%26email_token%3DANKPD24CUWKORS2STSGQMHDQL57RLA5CNFSM4EYBWCX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD725CCY%23issuecomment-536203531&data=02%7C01%7Croman.fleysher%40einstein.yu.edu%7Ca1a4cc38f43a4440e2a708d7442f5991%7C04c70eb48f2648079934e02e89266ad0%7C1%7C0%7C637052842480395917&sdata=xqCQliG34AIBYQ%2FYTAB7q3I5RfgsQZGKLwGMSEcwbkE%3D&reserved=0>, or mute the thread< https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FANKPD25IFXJSB6USGMZFYQ3QL57RLANCNFSM4EYBWCXQ&data=02%7C01%7Croman.fleysher%40einstein.yu.edu%7Ca1a4cc38f43a4440e2a708d7442f5991%7C04c70eb48f2648079934e02e89266ad0%7C1%7C0%7C637052842480405914&sdata=Ty%2FPv1JYtg35dSlVfAaMQsCq4WMC7RpQVBSrYmKuuCg%3D&reserved=0>. — You are receiving this because you commented. Reply to this email directly, view it on GitHub< #210?email_source=notifications&email_token=AAEVJM2C4QUFM7L7NVMJGK3QL6BB5A5CNFSM4EYBWCX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD725KMI#issuecomment-536204593>, or mute the thread< https://github.com/notifications/unsubscribe-auth/AAEVJM6GV4534DDBLFEFMKDQL6BB5ANCNFSM4EYBWCXQ>. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#210?email_source=notifications&email_token=ACFJU7OLCTA7DZHZVPUJOVDQL6BX7A5CNFSM4EYBWCX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD725OVA#issuecomment-536205140>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACFJU7JKPPFB2L3VLDK3FT3QL6BX7ANCNFSM4EYBWCXQ> .

muschellij2 · 2019-09-28T17:20:24Z

Is the code being run in ants or ANTsR? It wasn’t entirely clear from the message. I understand it’s an issue on this repo but just wanted to make sure

On Sat, Sep 28, 2019 at 1:16 PM dorianps ***@***.***> wrote: I am also interested in having a reproducible=TRUE in antsRegistration(). Would be good to have ANTsR itself take care of the needed arguments (seed and threads) to achieve it, and perhaps return the seed number for future use. On Sat, Sep 28, 2019, 12:36 PM Philip Cook ***@***.***> wrote: > I mean the commands that you ran > > Sent from my iPhone > > On Sep 28, 2019, at 12:30 PM, moskvich412 ***@***.*** > ***@***.***>> wrote: > > There is no code to share. I am running compiled code. I tried on mouse > and human, several pairs, several runs on each pair. They look similar, but > visibly different. I am not talking about 4th decimal place. Likely second > decimal. > > > > > -------- Original message -------- > From: Philip Cook ***@***.***<mailto: > ***@***.***>> > Date: 9/28/19 12:17 PM (GMT-05:00) > To: ANTsX/ANTsR ***@***.***<mailto: ***@***.***>> > > Cc: Roman Fleysher ***@***.***<mailto: > ***@***.***>>, Mention ***@***.*** > ***@***.***>> > Subject: Re: [ANTsX/ANTsR] antsRegistration does not produce equivalent > results (#210) > > I thought fixing the seed took care of this, but if you can share a code > and data I may be able to say more. > > > Sent from my iPhone > > > On Sep 28, 2019, at 11:36 AM, moskvich412 ***@***.*** > ***@***.***>> wrote: > > > > Putting philosophy aside, did i do the right thing to make ANTs > reproducible for the version we have? Is there else to try? Is remaining > variability intrinsic and can not be turned off? Is there another version > of ANTs which has the same morphism quality and reproducible? > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub< > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FANTsX%2FANTsR%2Fissues%2F210%3Femail_source%3Dnotifications%26email_token%3DANKPD24CUWKORS2STSGQMHDQL57RLA5CNFSM4EYBWCX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD725CCY%23issuecomment-536203531&data=02%7C01%7Croman.fleysher%40einstein.yu.edu%7Ca1a4cc38f43a4440e2a708d7442f5991%7C04c70eb48f2648079934e02e89266ad0%7C1%7C0%7C637052842480395917&sdata=xqCQliG34AIBYQ%2FYTAB7q3I5RfgsQZGKLwGMSEcwbkE%3D&reserved=0 >, > or mute the thread< > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FANKPD25IFXJSB6USGMZFYQ3QL57RLANCNFSM4EYBWCXQ&data=02%7C01%7Croman.fleysher%40einstein.yu.edu%7Ca1a4cc38f43a4440e2a708d7442f5991%7C04c70eb48f2648079934e02e89266ad0%7C1%7C0%7C637052842480405914&sdata=Ty%2FPv1JYtg35dSlVfAaMQsCq4WMC7RpQVBSrYmKuuCg%3D&reserved=0 >. > > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub< > #210?email_source=notifications&email_token=AAEVJM2C4QUFM7L7NVMJGK3QL6BB5A5CNFSM4EYBWCX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD725KMI#issuecomment-536204593 >, > or mute the thread< > https://github.com/notifications/unsubscribe-auth/AAEVJM6GV4534DDBLFEFMKDQL6BB5ANCNFSM4EYBWCXQ >. > > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > < #210?email_source=notifications&email_token=ACFJU7OLCTA7DZHZVPUJOVDQL6BX7A5CNFSM4EYBWCX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD725OVA#issuecomment-536205140 >, > or mute the thread > < https://github.com/notifications/unsubscribe-auth/ACFJU7JKPPFB2L3VLDK3FT3QL6BX7ANCNFSM4EYBWCXQ > > . > — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#210?email_source=notifications&email_token=AAIGPLUDN4FMEEPF3CBRVM3QL6GQBA5CNFSM4EYBWCX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD726IFA#issuecomment-536208404>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAIGPLXOSGDESBT4GAHA4JLQL6GQBANCNFSM4EYBWCXQ> .

-- Best, John

moskvich412 · 2019-09-28T20:46:13Z

I use this bash script to compute totalTransform.nii.gz:

export ANTS_RANDOM_SEED=10
export ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS=1

N3BiasFieldCorrection 3 input.nii.gz input.nii.gz 4
N3BiasFieldCorrection 3 template.nii.gz template.nii.gz 4

ANTS 3 -m CC[template.nii.gz,input.nii.gz,1,4] -t SyN[0.25]
-r Gauss[3,0] -o prefix -i 100x90x30 --use-Histogram-Matching
--number-of-affine-iterations 10000x10000x10000x10000x10000 --MI-option 32x16000

ComposeMultiTransform 3 totalTransform.nii.gz -R template.nii.gz prefixWarp.nii.gz prefixAffine.txt

The output morphism, totalTransform.nii.gz, is then applied using:

WarpImageMultiTransform 3 input.nii.gz morphedInput.nii.gz totalTransform.nii.gz -R totalTransform.nii.gz

I see that both N3BiasFieldCorrection and ANTS are running in a single thread. We are using version 2.3.1.

Comments on the choice of metrics, iterations, etc are also welcome. Maybe there is a better set.

stnava · 2019-09-28T20:57:00Z

See an example here ... looks like Mattes is not reproducible. Comments welcome.

It would be good to add Python and bash examples to above. Please contribute via pull request.

We don't recommend using the program called ANTS which is not maintained except perhaps to "get it to compile." The current tool is called antsRegistration.

ntustison · 2019-10-07T14:13:29Z

@muschellij2 Can you post an input image where this occurs? There's no explicitly random element in N3BiasFieldCorrection so I'm curious as to where this is happening.

muschellij2 · 2019-10-07T14:33:09Z

What I mean is: original image: input.nii.gz: iteration 1: N3BiasFieldCorrection 3 input.nii.gz input.nii.gz 4 (call this input_1.nii.gz) run warping and such N3BiasFieldCorrection 3 input.nii.gz input.nii.gz 4 Now this command is really: N3BiasFieldCorrection 3 input_1.nii.gz input_2.nii.gz 4 Note that the original input to N3 in iter 2 is not the same as iter 1. Best, John

…

On Mon, Oct 7, 2019 at 10:13 AM Nick Tustison ***@***.***> wrote: @muschellij2 <https://github.com/muschellij2> Can you post an input image where this occurs? There's no explicitly random element in N3BiasFieldCorrection so I'm curious as to where this is happening. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#210?email_source=notifications&email_token=AAIGPLRFM2SU5NYUMQ6YQITQNM7YXA5CNFSM4EYBWCX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAQPS7I#issuecomment-539031933>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAIGPLVWMCUOGHEQL4K6TG3QNM7YXANCNFSM4EYBWCXQ> .

ntustison · 2019-10-07T14:38:09Z

Okay, thanks for the clarification. I was worried since N4 has gone through changes over in the ITK GitHub repo, some of which concern overlapping elements with N3 and I wanted to make sure that I wasn't behind in my upkeep.

moskvich412 · 2019-10-07T15:21:21Z

I am sorry for not being clear. The set of commands I wrote were internal parts of my script, which first makes local copies of the input and template. Bias correction is not iteratively applied. I played with it separately and each iteration does change the image. There is no way to terminate this process because there is no good way to distinguish receive and transmit coils and T1 weighting in the context of MRI from the actual spatial variation in the image.

I also realized I was under influence of the older, version 1.9, manual which talked about the routines affected by randomness. I am glad the cause was my use of the deprecated commands which I will now fix and get reproducible results.

Upon upgrading our ANTS build we found that motion correction was no longer deterministic (#2642 (comment)). The ANTS maintainers suggested setting samplingStrategy=None (ANTsX/ANTs#976 (comment)) to get deterministic output, but they also warned that this will induce a variance-for-bias tradeoff: * https://github.com/ANTsX/ANTs/wiki/antsRegistration-reproducibility-issues * ANTsX/ANTsR#210 (comment) * Thévenaz, P., Bierlaire, M., & Unser, M. (2008). Halt on sampling for image registration based on mutual information. Sampling Theory in Signal and Image Processing, 7(2). <http://bigwww.epfl.ch/preprints/thevenaz0602p.pdf> Because we are hard-coding the sampling rate I removed it from ParamMoco entirely. If we want to support it properly we would have to pass .sampling_strategy and .sampling_percentage separately.

This makes the output deterministic, at the cost of running unnecessarily slowly. The order of the floating point sums used internally is numerically unstable. See https://github.com/ANTsX/ANTs/wiki/antsRegistration-reproducibility-issues ANTsX/ANTs#444 (comment) ANTsX/ANTsR#210 (comment)

Upon upgrading our ANTS build we found that motion correction was no longer deterministic (#2642 (comment)). The ANTS maintainers suggested setting samplingStrategy=None (ANTsX/ANTs#976 (comment)) to get deterministic output, but they also warned that this will induce a variance-for-bias tradeoff: * https://github.com/ANTsX/ANTs/wiki/antsRegistration-reproducibility-issues * ANTsX/ANTsR#210 (comment) * Thévenaz, P., Bierlaire, M., & Unser, M. (2008). Halt on sampling for image registration based on mutual information. Sampling Theory in Signal and Image Processing, 7(2). <http://bigwww.epfl.ch/preprints/thevenaz0602p.pdf> Because we are hard-coding the sampling rate I removed it from ParamMoco entirely. If we want to support it properly we would have to pass .sampling_strategy and .sampling_percentage separately.

This is also to make the output deterministic, at the cost of running slow. It turned out that using dense sampling wasn't enough; there was still some numerical instability that came from the order of addition: * https://github.com/ANTsX/ANTs/wiki/antsRegistration-reproducibility-issues#variance-due-to-floating-point-precision-errors For some reason it only appeared on OS X, and only about 10% of the time, and never on Linux. I [showed](#2642 (comment)) that the instability in isct_antsSliceRegularizedRegistration did exist on Linux, so something still unknown about how we call it was hiding it there. See: * https://github.com/ANTsX/ANTs/wiki/antsRegistration-reproducibility-issues * ANTsX/ANTs#444 (comment) * ANTsX/ANTsR#210 (comment)

This is also to make the output deterministic, at the cost of running slow. It turned out that using dense sampling wasn't enough; there was still some numerical instability that came from the order of addition: * https://github.com/ANTsX/ANTs/wiki/antsRegistration-reproducibility-issues#variance-due-to-floating-point-precision-errors For some reason it only appeared on OS X, and only about 10% of the time, and never on Linux. I [showed](#2642 (comment)) that the instability in isct_antsSliceRegularizedRegistration did exist on Linux, so something still unknown about how we call it was hiding it there. This was actually supposed to be in place already but the code had atrophied, so all this does is fix it up. See: * https://github.com/ANTsX/ANTs/wiki/antsRegistration-reproducibility-issues * ANTsX/ANTs#444 (comment) * ANTsX/ANTsR#210 (comment)

Upon upgrading our ANTS build we found that motion correction was no longer deterministic (#2642 (comment)). The ANTS maintainers suggested setting samplingStrategy=None (ANTsX/ANTs#976 (comment)) to get deterministic output, but they also warned that this will induce a variance-for-bias tradeoff: * https://github.com/ANTsX/ANTs/wiki/antsRegistration-reproducibility-issues * ANTsX/ANTsR#210 (comment) * Thévenaz, P., Bierlaire, M., & Unser, M. (2008). Halt on sampling for image registration based on mutual information. Sampling Theory in Signal and Image Processing, 7(2). <http://bigwww.epfl.ch/preprints/thevenaz0602p.pdf> Because we are hard-coding the sampling rate I removed it from ParamMoco entirely. If we want to support it properly we would have to pass .sampling_strategy and .sampling_percentage separately.

This is also to make the output deterministic, at the cost of running slow. It turned out that using dense sampling wasn't enough; there was still some numerical instability that came from the order of addition: * https://github.com/ANTsX/ANTs/wiki/antsRegistration-reproducibility-issues#variance-due-to-floating-point-precision-errors For some reason it only appeared on OS X, and only about 10% of the time, and never on Linux. I [showed](#2642 (comment)) that the instability in isct_antsSliceRegularizedRegistration did exist on Linux, so something still unknown about how we call it was hiding it there. This was actually supposed to be in place already but the code had atrophied, so all this does is fix it up. See: * https://github.com/ANTsX/ANTs/wiki/antsRegistration-reproducibility-issues * ANTsX/ANTs#444 (comment) * ANTsX/ANTsR#210 (comment)

Upon upgrading our ANTS build we found that motion correction was no longer deterministic (#2642 (comment)). The ANTS maintainers suggested setting samplingStrategy=None (ANTsX/ANTs#976 (comment)) to get deterministic output, but they also warned that this will induce a variance-for-bias tradeoff: * https://github.com/ANTsX/ANTs/wiki/antsRegistration-reproducibility-issues * ANTsX/ANTsR#210 (comment) * Thévenaz, P., Bierlaire, M., & Unser, M. (2008). Halt on sampling for image registration based on mutual information. Sampling Theory in Signal and Image Processing, 7(2). <http://bigwww.epfl.ch/preprints/thevenaz0602p.pdf> Because we are hard-coding the sampling rate I removed it from ParamMoco entirely. If we want to support it properly we would have to pass .sampling_strategy and .sampling_percentage separately.

This is also to make the output deterministic, at the cost of running slow. It turned out that using dense sampling wasn't enough; there was still some numerical instability that came from the order of addition: * https://github.com/ANTsX/ANTs/wiki/antsRegistration-reproducibility-issues#variance-due-to-floating-point-precision-errors For some reason it only appeared on OS X, and only about 10% of the time, and never on Linux. I [showed](#2642 (comment)) that the instability in isct_antsSliceRegularizedRegistration did exist on Linux, so something still unknown about how we call it was hiding it there. This was actually supposed to be in place already but the code had atrophied, so all this does is fix it up. See: * https://github.com/ANTsX/ANTs/wiki/antsRegistration-reproducibility-issues * ANTsX/ANTs#444 (comment) * ANTsX/ANTsR#210 (comment)

Upon upgrading our ANTS build we found that motion correction was no longer deterministic (#2642 (comment)). The ANTS maintainers suggested setting samplingStrategy=None (ANTsX/ANTs#976 (comment)) to get deterministic output, but they also warned that this will induce a variance-for-bias tradeoff: * https://github.com/ANTsX/ANTs/wiki/antsRegistration-reproducibility-issues * ANTsX/ANTsR#210 (comment) * Thévenaz, P., Bierlaire, M., & Unser, M. (2008). Halt on sampling for image registration based on mutual information. Sampling Theory in Signal and Image Processing, 7(2). <http://bigwww.epfl.ch/preprints/thevenaz0602p.pdf> Because we are hard-coding the sampling rate I removed it from ParamMoco entirely. If we want to support it properly we would have to pass .sampling_strategy and .sampling_percentage separately.

This is also to make the output deterministic, at the cost of running slow. It turned out that using dense sampling wasn't enough; there was still some numerical instability that came from the order of addition: * https://github.com/ANTsX/ANTs/wiki/antsRegistration-reproducibility-issues#variance-due-to-floating-point-precision-errors For some reason it only appeared on OS X, and only about 10% of the time, and never on Linux. I [showed](#2642 (comment)) that the instability in isct_antsSliceRegularizedRegistration did exist on Linux, so something still unknown about how we call it was hiding it there. This was actually supposed to be in place already but the code had atrophied, so all this does is fix it up. See: * https://github.com/ANTsX/ANTs/wiki/antsRegistration-reproducibility-issues * ANTsX/ANTs#444 (comment) * ANTsX/ANTsR#210 (comment)

Upon upgrading our ANTS build we found that motion correction was no longer deterministic (#2642 (comment)). The ANTS maintainers suggested setting samplingStrategy=None (ANTsX/ANTs#976 (comment)) to get deterministic output, but they also warned that this will induce a variance-for-bias tradeoff: * https://github.com/ANTsX/ANTs/wiki/antsRegistration-reproducibility-issues * ANTsX/ANTsR#210 (comment) * Thévenaz, P., Bierlaire, M., & Unser, M. (2008). Halt on sampling for image registration based on mutual information. Sampling Theory in Signal and Image Processing, 7(2). <http://bigwww.epfl.ch/preprints/thevenaz0602p.pdf> I think ideally moco.py would thinly wrap antsRegistration, and expose both .sampling_strategy and .sampling_percentage directly, but until now we've always hardcoded 'samplingStrategy=Regular' so in the interest of not rocking the boat too much all I did was add a 'sampling=None' case, *and set it as the default* Updated version re feedback: #2642 (comment)

Upon upgrading our ANTS build we found that motion correction was no longer deterministic (#2642 (comment)). The ANTS maintainers suggested setting samplingStrategy=None (ANTsX/ANTs#976 (comment)) to get deterministic output, but they also warned that this will induce a variance-for-bias tradeoff: * https://github.com/ANTsX/ANTs/wiki/antsRegistration-reproducibility-issues * ANTsX/ANTsR#210 (comment) * Thévenaz, P., Bierlaire, M., & Unser, M. (2008). Halt on sampling for image registration based on mutual information. Sampling Theory in Signal and Image Processing, 7(2). <http://bigwww.epfl.ch/preprints/thevenaz0602p.pdf> Because we are hard-coding the sampling rate I removed it from ParamMoco entirely. If we want to support it properly we would have to pass .sampling_strategy and .sampling_percentage separately.

This is also to make the output deterministic, at the cost of running slow. It turned out that using dense sampling wasn't enough; there was still some numerical instability that came from the order of addition: * https://github.com/ANTsX/ANTs/wiki/antsRegistration-reproducibility-issues#variance-due-to-floating-point-precision-errors For some reason it only appeared on OS X, and only about 10% of the time, and never on Linux. I [showed](#2642 (comment)) that the instability in isct_antsSliceRegularizedRegistration did exist on Linux, so something still unknown about how we call it was hiding it there. This was actually supposed to be in place already but the code had atrophied, so all this does is fix it up. See: * https://github.com/ANTsX/ANTs/wiki/antsRegistration-reproducibility-issues * ANTsX/ANTs#444 (comment) * ANTsX/ANTsR#210 (comment)

Upon upgrading our ANTS build we found that motion correction was no longer deterministic (#2642 (comment)). The ANTS maintainers suggested setting samplingStrategy=None (ANTsX/ANTs#976 (comment)) to get deterministic output, but they also warned that this will induce a variance-for-bias tradeoff: * https://github.com/ANTsX/ANTs/wiki/antsRegistration-reproducibility-issues * ANTsX/ANTsR#210 (comment) * Thévenaz, P., Bierlaire, M., & Unser, M. (2008). Halt on sampling for image registration based on mutual information. Sampling Theory in Signal and Image Processing, 7(2). <http://bigwww.epfl.ch/preprints/thevenaz0602p.pdf> I think ideally moco.py would thinly wrap antsRegistration, and expose both .sampling_strategy and .sampling_percentage directly, but until now we've always hardcoded 'samplingStrategy=Regular' so in the interest of not rocking the boat too much all I did was add a 'sampling=None' case, *and set it as the default* Updated version re feedback: #2642 (comment)

muschellij2 · 2020-08-06T16:13:16Z

So revisiting this a bit again, I think there is a sneaky bug somewhere for this. I don't believe this is fixed.

Here are 3 different scenarios, setting environment variables before loading (like .Renviron would do), setting it after, and then setting it after some ANTsR operation has been completed.

Setting the Seed before `ANTsR` Loaded - Works

Sys.setenv(ANTS_RANDOM_SEED = 123)
Sys.setenv(ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS = 1)
library( ANTsR )
#> Loading required package: ANTsRCore
#> 
#> Attaching package: 'ANTsRCore'
#> The following objects are masked from 'package:stats':
#> 
#>     sd, var
#> The following objects are masked from 'package:base':
#> 
#>     all, any, apply, max, min, prod, range, sum

fi <- antsImageRead(getANTsRData("r16") )
mi <- antsImageRead(getANTsRData("r64") )

tx = "SyN"
print(Sys.getenv("ANTS_RANDOM_SEED"))
#> [1] "123"
mytx1 <- antsRegistration(fixed=fi, mi, typeofTransform = tx)
mytx2 <- antsRegistration(fixed=fi, mi, typeofTransform = tx)
print(sum(mytx1$warpedmovout - mytx2$warpedmovout))
#> [1] 0
max(abs(as.array(mytx1$warpedmovout) - as.array(mytx2$warpedmovout)))
#> [1] 0

^{Created on 2020-08-06 by the reprex package (v0.3.0)}

Session info

devtools::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value                       
#>  version  R version 4.0.0 (2020-04-24)
#>  os       macOS Mojave 10.14.6        
#>  system   x86_64, darwin17.0          
#>  ui       X11                         
#>  language (EN)                        
#>  collate  en_US.UTF-8                 
#>  ctype    en_US.UTF-8                 
#>  tz       America/New_York            
#>  date     2020-08-06                  
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package     * version    date       lib source                                
#>  ANTsR       * 0.5.6.1    2020-06-01 [1] Github (ANTsX/ANTsR@9c7c9b7)          
#>  ANTsRCore   * 0.7.4.6    2020-07-07 [1] Github (muschellij2/ANTsRCore@61c37a1)
#>  assertthat    0.2.1      2019-03-21 [1] CRAN (R 4.0.0)                        
#>  backports     1.1.8      2020-06-17 [1] CRAN (R 4.0.0)                        
#>  callr         3.4.3      2020-03-28 [1] CRAN (R 4.0.0)                        
#>  cli           2.0.2      2020-02-28 [1] CRAN (R 4.0.0)                        
#>  crayon        1.3.4      2017-09-16 [1] CRAN (R 4.0.0)                        
#>  desc          1.2.0      2020-06-01 [1] Github (muschellij2/desc@b0c374f)     
#>  devtools      2.3.0.9000 2020-06-01 [1] Github (hadley/devtools@202ea81)      
#>  digest        0.6.25     2020-02-23 [1] CRAN (R 4.0.0)                        
#>  ellipsis      0.3.1      2020-05-15 [1] CRAN (R 4.0.0)                        
#>  evaluate      0.14       2019-05-28 [1] CRAN (R 4.0.0)                        
#>  fansi         0.4.1      2020-01-08 [1] CRAN (R 4.0.0)                        
#>  fs            1.4.2      2020-06-30 [1] CRAN (R 4.0.0)                        
#>  glue          1.4.1      2020-05-13 [1] CRAN (R 4.0.0)                        
#>  highr         0.8        2019-03-20 [1] CRAN (R 4.0.0)                        
#>  htmltools     0.5.0      2020-06-16 [1] CRAN (R 4.0.0)                        
#>  ITKR          0.5.3.2.0  2020-06-01 [1] Github (stnava/ITKR@9bdd5f8)          
#>  knitr         1.29.3     2020-07-24 [1] Github (muschellij2/knitr@55195ad)    
#>  lattice       0.20-41    2020-04-02 [1] CRAN (R 4.0.0)                        
#>  magrittr      1.5        2014-11-22 [1] CRAN (R 4.0.0)                        
#>  Matrix        1.2-18     2019-11-27 [1] CRAN (R 4.0.0)                        
#>  memoise       1.1.0      2017-04-21 [1] CRAN (R 4.0.0)                        
#>  pkgbuild      1.0.8      2020-05-07 [1] CRAN (R 4.0.0)                        
#>  pkgload       1.1.0      2020-05-29 [1] CRAN (R 4.0.0)                        
#>  prettyunits   1.1.1      2020-01-24 [1] CRAN (R 4.0.0)                        
#>  processx      3.4.3      2020-07-05 [1] CRAN (R 4.0.0)                        
#>  ps            1.3.3      2020-05-08 [1] CRAN (R 4.0.0)                        
#>  R6            2.4.1      2019-11-12 [1] CRAN (R 4.0.0)                        
#>  Rcpp          1.0.5      2020-07-06 [1] CRAN (R 4.0.0)                        
#>  RcppEigen     0.3.3.7.0  2019-11-16 [1] CRAN (R 4.0.0)                        
#>  remotes       2.1.1      2020-02-15 [1] CRAN (R 4.0.0)                        
#>  rlang         0.4.6.9000 2020-07-08 [1] Github (r-lib/rlang@f3cdf54)          
#>  rmarkdown     2.3        2020-06-18 [1] CRAN (R 4.0.0)                        
#>  rprojroot     1.3-2      2018-01-03 [1] CRAN (R 4.0.0)                        
#>  sessioninfo   1.1.1      2018-11-05 [1] CRAN (R 4.0.0)                        
#>  stringi       1.4.6      2020-02-17 [1] CRAN (R 4.0.0)                        
#>  stringr       1.4.0      2019-02-10 [1] CRAN (R 4.0.0)                        
#>  testthat      2.3.2      2020-03-02 [1] CRAN (R 4.0.0)                        
#>  usethis       1.6.1.9000 2020-07-06 [1] Github (r-lib/usethis@c1cb6d9)        
#>  withr         2.2.0      2020-04-20 [1] CRAN (R 4.0.0)                        
#>  xfun          0.15       2020-06-21 [1] CRAN (R 4.0.0)                        
#>  yaml          2.2.1      2020-02-01 [1] CRAN (R 4.0.0)                        
#> 
#> [1] /Library/Frameworks/R.framework/Versions/4.0/Resources/library

Setting after Loading `ANTsR` - Works

library( ANTsR )
#> Loading required package: ANTsRCore
#> 
#> Attaching package: 'ANTsRCore'
#> The following objects are masked from 'package:stats':
#> 
#>     sd, var
#> The following objects are masked from 'package:base':
#> 
#>     all, any, apply, max, min, prod, range, sum

Sys.setenv(ANTS_RANDOM_SEED = 123)
Sys.setenv(ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS = 1)

fi <- antsImageRead(getANTsRData("r16") )
mi <- antsImageRead(getANTsRData("r64") )

tx = "SyN"
print(Sys.getenv("ANTS_RANDOM_SEED"))
#> [1] "123"
mytx1 <- antsRegistration(fixed=fi, mi, typeofTransform = tx)
mytx2 <- antsRegistration(fixed=fi, mi, typeofTransform = tx)
print(sum(mytx1$warpedmovout - mytx2$warpedmovout))
#> [1] 0
max(abs(as.array(mytx1$warpedmovout) - as.array(mytx2$warpedmovout)))
#> [1] 0

^{Created on 2020-08-06 by the reprex package (v0.3.0)}

Session info

devtools::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value                       
#>  version  R version 4.0.0 (2020-04-24)
#>  os       macOS Mojave 10.14.6        
#>  system   x86_64, darwin17.0          
#>  ui       X11                         
#>  language (EN)                        
#>  collate  en_US.UTF-8                 
#>  ctype    en_US.UTF-8                 
#>  tz       America/New_York            
#>  date     2020-08-06                  
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package     * version    date       lib source                                
#>  ANTsR       * 0.5.6.1    2020-06-01 [1] Github (ANTsX/ANTsR@9c7c9b7)          
#>  ANTsRCore   * 0.7.4.6    2020-07-07 [1] Github (muschellij2/ANTsRCore@61c37a1)
#>  assertthat    0.2.1      2019-03-21 [1] CRAN (R 4.0.0)                        
#>  backports     1.1.8      2020-06-17 [1] CRAN (R 4.0.0)                        
#>  callr         3.4.3      2020-03-28 [1] CRAN (R 4.0.0)                        
#>  cli           2.0.2      2020-02-28 [1] CRAN (R 4.0.0)                        
#>  crayon        1.3.4      2017-09-16 [1] CRAN (R 4.0.0)                        
#>  desc          1.2.0      2020-06-01 [1] Github (muschellij2/desc@b0c374f)     
#>  devtools      2.3.0.9000 2020-06-01 [1] Github (hadley/devtools@202ea81)      
#>  digest        0.6.25     2020-02-23 [1] CRAN (R 4.0.0)                        
#>  ellipsis      0.3.1      2020-05-15 [1] CRAN (R 4.0.0)                        
#>  evaluate      0.14       2019-05-28 [1] CRAN (R 4.0.0)                        
#>  fansi         0.4.1      2020-01-08 [1] CRAN (R 4.0.0)                        
#>  fs            1.4.2      2020-06-30 [1] CRAN (R 4.0.0)                        
#>  glue          1.4.1      2020-05-13 [1] CRAN (R 4.0.0)                        
#>  highr         0.8        2019-03-20 [1] CRAN (R 4.0.0)                        
#>  htmltools     0.5.0      2020-06-16 [1] CRAN (R 4.0.0)                        
#>  ITKR          0.5.3.2.0  2020-06-01 [1] Github (stnava/ITKR@9bdd5f8)          
#>  knitr         1.29.3     2020-07-24 [1] Github (muschellij2/knitr@55195ad)    
#>  lattice       0.20-41    2020-04-02 [1] CRAN (R 4.0.0)                        
#>  magrittr      1.5        2014-11-22 [1] CRAN (R 4.0.0)                        
#>  Matrix        1.2-18     2019-11-27 [1] CRAN (R 4.0.0)                        
#>  memoise       1.1.0      2017-04-21 [1] CRAN (R 4.0.0)                        
#>  pkgbuild      1.0.8      2020-05-07 [1] CRAN (R 4.0.0)                        
#>  pkgload       1.1.0      2020-05-29 [1] CRAN (R 4.0.0)                        
#>  prettyunits   1.1.1      2020-01-24 [1] CRAN (R 4.0.0)                        
#>  processx      3.4.3      2020-07-05 [1] CRAN (R 4.0.0)                        
#>  ps            1.3.3      2020-05-08 [1] CRAN (R 4.0.0)                        
#>  R6            2.4.1      2019-11-12 [1] CRAN (R 4.0.0)                        
#>  Rcpp          1.0.5      2020-07-06 [1] CRAN (R 4.0.0)                        
#>  RcppEigen     0.3.3.7.0  2019-11-16 [1] CRAN (R 4.0.0)                        
#>  remotes       2.1.1      2020-02-15 [1] CRAN (R 4.0.0)                        
#>  rlang         0.4.6.9000 2020-07-08 [1] Github (r-lib/rlang@f3cdf54)          
#>  rmarkdown     2.3        2020-06-18 [1] CRAN (R 4.0.0)                        
#>  rprojroot     1.3-2      2018-01-03 [1] CRAN (R 4.0.0)                        
#>  sessioninfo   1.1.1      2018-11-05 [1] CRAN (R 4.0.0)                        
#>  stringi       1.4.6      2020-02-17 [1] CRAN (R 4.0.0)                        
#>  stringr       1.4.0      2019-02-10 [1] CRAN (R 4.0.0)                        
#>  testthat      2.3.2      2020-03-02 [1] CRAN (R 4.0.0)                        
#>  usethis       1.6.1.9000 2020-07-06 [1] Github (r-lib/usethis@c1cb6d9)        
#>  withr         2.2.0      2020-04-20 [1] CRAN (R 4.0.0)                        
#>  xfun          0.15       2020-06-21 [1] CRAN (R 4.0.0)                        
#>  yaml          2.2.1      2020-02-01 [1] CRAN (R 4.0.0)                        
#> 
#> [1] /Library/Frameworks/R.framework/Versions/4.0/Resources/library

Setting Seed after reading data - FAILS

library( ANTsR )
#> Loading required package: ANTsRCore
#> 
#> Attaching package: 'ANTsRCore'
#> The following objects are masked from 'package:stats':
#> 
#>     sd, var
#> The following objects are masked from 'package:base':
#> 
#>     all, any, apply, max, min, prod, range, sum

fi <- antsImageRead(getANTsRData("r16") )
mi <- antsImageRead(getANTsRData("r64") )

Sys.setenv(ANTS_RANDOM_SEED = 123)
Sys.setenv(ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS = 1)
tx = "SyN"
print(Sys.getenv("ANTS_RANDOM_SEED"))
#> [1] "123"
mytx1 <- antsRegistration(fixed=fi, mi, typeofTransform = tx)
mytx2 <- antsRegistration(fixed=fi, mi, typeofTransform = tx)
print(sum(mytx1$warpedmovout - mytx2$warpedmovout))
#> [1] 117.2713
max(abs(as.array(mytx1$warpedmovout) - as.array(mytx2$warpedmovout)))
#> [1] 103.0909

^{Created on 2020-08-06 by the reprex package (v0.3.0)}

Session info

devtools::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value                       
#>  version  R version 4.0.0 (2020-04-24)
#>  os       macOS Mojave 10.14.6        
#>  system   x86_64, darwin17.0          
#>  ui       X11                         
#>  language (EN)                        
#>  collate  en_US.UTF-8                 
#>  ctype    en_US.UTF-8                 
#>  tz       America/New_York            
#>  date     2020-08-06                  
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package     * version    date       lib source                                
#>  ANTsR       * 0.5.6.1    2020-06-01 [1] Github (ANTsX/ANTsR@9c7c9b7)          
#>  ANTsRCore   * 0.7.4.6    2020-07-07 [1] Github (muschellij2/ANTsRCore@61c37a1)
#>  assertthat    0.2.1      2019-03-21 [1] CRAN (R 4.0.0)                        
#>  backports     1.1.8      2020-06-17 [1] CRAN (R 4.0.0)                        
#>  callr         3.4.3      2020-03-28 [1] CRAN (R 4.0.0)                        
#>  cli           2.0.2      2020-02-28 [1] CRAN (R 4.0.0)                        
#>  crayon        1.3.4      2017-09-16 [1] CRAN (R 4.0.0)                        
#>  desc          1.2.0      2020-06-01 [1] Github (muschellij2/desc@b0c374f)     
#>  devtools      2.3.0.9000 2020-06-01 [1] Github (hadley/devtools@202ea81)      
#>  digest        0.6.25     2020-02-23 [1] CRAN (R 4.0.0)                        
#>  ellipsis      0.3.1      2020-05-15 [1] CRAN (R 4.0.0)                        
#>  evaluate      0.14       2019-05-28 [1] CRAN (R 4.0.0)                        
#>  fansi         0.4.1      2020-01-08 [1] CRAN (R 4.0.0)                        
#>  fs            1.4.2      2020-06-30 [1] CRAN (R 4.0.0)                        
#>  glue          1.4.1      2020-05-13 [1] CRAN (R 4.0.0)                        
#>  highr         0.8        2019-03-20 [1] CRAN (R 4.0.0)                        
#>  htmltools     0.5.0      2020-06-16 [1] CRAN (R 4.0.0)                        
#>  ITKR          0.5.3.2.0  2020-06-01 [1] Github (stnava/ITKR@9bdd5f8)          
#>  knitr         1.29.3     2020-07-24 [1] Github (muschellij2/knitr@55195ad)    
#>  lattice       0.20-41    2020-04-02 [1] CRAN (R 4.0.0)                        
#>  magrittr      1.5        2014-11-22 [1] CRAN (R 4.0.0)                        
#>  Matrix        1.2-18     2019-11-27 [1] CRAN (R 4.0.0)                        
#>  memoise       1.1.0      2017-04-21 [1] CRAN (R 4.0.0)                        
#>  pkgbuild      1.0.8      2020-05-07 [1] CRAN (R 4.0.0)                        
#>  pkgload       1.1.0      2020-05-29 [1] CRAN (R 4.0.0)                        
#>  prettyunits   1.1.1      2020-01-24 [1] CRAN (R 4.0.0)                        
#>  processx      3.4.3      2020-07-05 [1] CRAN (R 4.0.0)                        
#>  ps            1.3.3      2020-05-08 [1] CRAN (R 4.0.0)                        
#>  R6            2.4.1      2019-11-12 [1] CRAN (R 4.0.0)                        
#>  Rcpp          1.0.5      2020-07-06 [1] CRAN (R 4.0.0)                        
#>  RcppEigen     0.3.3.7.0  2019-11-16 [1] CRAN (R 4.0.0)                        
#>  remotes       2.1.1      2020-02-15 [1] CRAN (R 4.0.0)                        
#>  rlang         0.4.6.9000 2020-07-08 [1] Github (r-lib/rlang@f3cdf54)          
#>  rmarkdown     2.3        2020-06-18 [1] CRAN (R 4.0.0)                        
#>  rprojroot     1.3-2      2018-01-03 [1] CRAN (R 4.0.0)                        
#>  sessioninfo   1.1.1      2018-11-05 [1] CRAN (R 4.0.0)                        
#>  stringi       1.4.6      2020-02-17 [1] CRAN (R 4.0.0)                        
#>  stringr       1.4.0      2019-02-10 [1] CRAN (R 4.0.0)                        
#>  testthat      2.3.2      2020-03-02 [1] CRAN (R 4.0.0)                        
#>  usethis       1.6.1.9000 2020-07-06 [1] Github (r-lib/usethis@c1cb6d9)        
#>  withr         2.2.0      2020-04-20 [1] CRAN (R 4.0.0)                        
#>  xfun          0.15       2020-06-21 [1] CRAN (R 4.0.0)                        
#>  yaml          2.2.1      2020-02-01 [1] CRAN (R 4.0.0)                        
#> 
#> [1] /Library/Frameworks/R.framework/Versions/4.0/Resources/library

Example video demonstrating this: https://www.youtube.com/watch?v=tyz5Pbp-35o&feature=youtu.be

muschellij2 · 2020-09-30T18:01:43Z

Can anyone confirm this on their machine? @adigherman ? @cookpa @stnava

cookpa · 2020-09-30T18:06:55Z

Will try but may take some days

giadadam · 2024-03-26T11:08:40Z

Hello! I would like to store the seed that was used in a specific run, so that we are able to reproduce that data in the future. How can this be done? Thanks in advance!

stnava closed this as completed Jun 3, 2018

This was referenced Jul 12, 2018

Randomness in calculation of FD/Motion Parameters #226

Closed

Reproducible kmeans segmentation ANTsX/ANTsRCore#51

Closed

stnava reopened this Sep 28, 2019

mgxd mentioned this issue Apr 16, 2020

ENH: Keep track and allow reuse of random seeds with the config module nipreps/fmriprep#2086

Merged

kousu mentioned this issue Apr 25, 2020

Updated ANTs binaries to solve slow processing in some systems spinalcordtoolbox/spinalcordtoolbox#2642

Merged

muschellij2 reopened this Aug 6, 2020

muschellij2 mentioned this issue Aug 8, 2020

Verbose registration ANTsX/ANTs#1063

Closed

leonardocerliani mentioned this issue Jan 5, 2021

setting ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS from within R #336

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

antsRegistration does not produce equivalent results #210

antsRegistration does not produce equivalent results #210

dmirman-zz commented Mar 29, 2018

ntustison commented Mar 29, 2018

stnava commented Mar 29, 2018 via email

muschellij2 commented Mar 29, 2018 •

edited

Loading

dmirman-zz commented Mar 29, 2018

stnava commented Mar 29, 2018 via email

dmirman-zz commented Mar 29, 2018

cookpa commented Mar 29, 2018

dmirman-zz commented Mar 29, 2018

ntustison commented Mar 29, 2018

dorianps commented Mar 30, 2018 via email

stnava commented Mar 30, 2018 via email

ntustison commented Mar 30, 2018

dmirman-zz commented Apr 2, 2018

stnava commented Apr 2, 2018 via email

stnava commented Jun 3, 2018

moskvich412 commented Sep 27, 2019

ntustison commented Sep 28, 2019

moskvich412 commented Sep 28, 2019 via email

ntustison commented Sep 28, 2019

cookpa commented Sep 28, 2019 via email

moskvich412 commented Sep 28, 2019 via email •

edited

Loading

cookpa commented Sep 28, 2019 via email

dorianps commented Sep 28, 2019 via email

muschellij2 commented Sep 28, 2019 via email

moskvich412 commented Sep 28, 2019

stnava commented Sep 28, 2019 •

edited

Loading

ntustison commented Oct 7, 2019

muschellij2 commented Oct 7, 2019 via email

ntustison commented Oct 7, 2019

moskvich412 commented Oct 7, 2019

muschellij2 commented Aug 6, 2020

muschellij2 commented Sep 30, 2020

cookpa commented Sep 30, 2020

giadadam commented Mar 26, 2024

antsRegistration does not produce equivalent results #210

antsRegistration does not produce equivalent results #210

Comments

dmirman-zz commented Mar 29, 2018

ntustison commented Mar 29, 2018

stnava commented Mar 29, 2018 via email

muschellij2 commented Mar 29, 2018 • edited Loading

dmirman-zz commented Mar 29, 2018

stnava commented Mar 29, 2018 via email

dmirman-zz commented Mar 29, 2018

cookpa commented Mar 29, 2018

dmirman-zz commented Mar 29, 2018

ntustison commented Mar 29, 2018

dorianps commented Mar 30, 2018 via email

stnava commented Mar 30, 2018 via email

ntustison commented Mar 30, 2018

dmirman-zz commented Apr 2, 2018

stnava commented Apr 2, 2018 via email

stnava commented Jun 3, 2018

moskvich412 commented Sep 27, 2019

ntustison commented Sep 28, 2019

moskvich412 commented Sep 28, 2019 via email

ntustison commented Sep 28, 2019

cookpa commented Sep 28, 2019 via email

moskvich412 commented Sep 28, 2019 via email • edited Loading

cookpa commented Sep 28, 2019 via email

dorianps commented Sep 28, 2019 via email

muschellij2 commented Sep 28, 2019 via email

moskvich412 commented Sep 28, 2019

stnava commented Sep 28, 2019 • edited Loading

ntustison commented Oct 7, 2019

muschellij2 commented Oct 7, 2019 via email

ntustison commented Oct 7, 2019

moskvich412 commented Oct 7, 2019

muschellij2 commented Aug 6, 2020

Setting the Seed before ANTsR Loaded - Works

Setting after Loading ANTsR - Works

Setting Seed after reading data - FAILS

muschellij2 commented Sep 30, 2020

cookpa commented Sep 30, 2020

giadadam commented Mar 26, 2024

muschellij2 commented Mar 29, 2018 •

edited

Loading

moskvich412 commented Sep 28, 2019 via email •

edited

Loading

stnava commented Sep 28, 2019 •

edited

Loading

Setting the Seed before `ANTsR` Loaded - Works

Setting after Loading `ANTsR` - Works