LANDIS II model with DynamicGrids.jl

[VLucet]

Hi and thanks for this awesome package! I am currently looking for efficient alternatives to the forest landscape model LANDIS-II, that would not be reliant on C# like LANDIS is.

On your issue board I found references to the LANDIS models (in #114 and #145). I was wondering to what extent DynamicGrids.jl could allow to recreate a full-fledge LANDIS model and whether this had been done (if not fully, maybe in part) somewhere? Would that warrant a separate package?

[rafaqz]

Yes, we've been thinking about landis a lot. I've already started writing the components! There's also a paragraph on this in our in-review paper for DynamicGrids.jl.

But DynamicGrids.jl could easily provide the engine for a more flexible and faster replacement for LANDIS II/LANDIS Pro etc, running models on GPUs. Even on CPU it should be significantly faster as optimizations are so easy to write in Julia. The other major benefits are data input is much more flexible, and you could mix and match landis style models with other custom models and e.g. Dispersal.jl organism dispersal models.

So do you want to collaborate on a package? I was thinking I was going to start one and call it Landscape.jl. But I could really use some help from someone with domain knowledge and experience with LANDIS. I've read a lot of papers, but I haven't actually used it.

One main thing that's missing is a fast StaticBitMatrix implementation to represent a size/species matrix with low memory overheads. We probably need to write that package as well, but it's not too much work I don't think.

(BTW yes, as with Dispersal.jl it's good to keep domain specific models separate to the framework code in DynamicGrids.jl, that way Landscape.jl could be mostly formulations with no low-level machinery to distract readers/users)

[VLucet]

There's also a paragraph on this in our in-review paper for DynamicGrids.jl.

Is this paper accessible? Would love to read it.

The other major benefits are data input is much more flexible, and you could mix and match landis style models with other custom models and e.g. Dispersal.jl organism dispersal models.

Yes the mix and match idea is what pushed me to post an issue, I think this is where Julia has an edge, as I think one of the weakness of LANDIS is that it is not so easy to combine with other models (at least, from my experience).

So do you want to collaborate on a package? I was thinking I was going to start one and call it Landscape.jl. But I could really use some help from someone with domain knowledge and experience with LANDIS. I've read a lot of papers, but I haven't actually used it.

I would sincerelly love to help, as I have some experience with LANDIS and Julia. However, my knowledge of (1) How LANDIS is implemented under the hood and (2) Package development in Julia remains very limited (for now!). I was hoping the thing already existed in Julia so I could conveniently learn both at the same time. For more domain knowlegde I would maybe ping @Klemet who I know wrote a LANDIS extension and might be inclined to be involved.

Personnaly, I would be happy to take on small tasks so I can learn more, but I don't have a lot of relevant experience developping in Julia, so I might not be the ideal domain knowledge partner for this project. That being said, I would be very much interested in following the project, be an early test user and possibly collaborate on a paper for the package if that sounds like something you'd like todo, as I have use cases in mind that could be used as an example. I might know people who would be interested at NRCAN/ECCC as well, and can follow up on that.

One main thing that's missing is a fast StaticBitMatrix implementation to represent a size/species matrix with low memory overheads. We probably need to write that package as well, but it's not too much work I don't think.

Isn't this already possible with StaticArrays? Or maybe related to JuliaArrays/StaticArrays.jl#412?

(BTW yes, as with Dispersal.jl it's good to keep domain specific models separate to the framework code in DynamicGrids.jl, that way Landscape.jl could be mostly formulations with no low-level machinery to distract readers/users)

Yes, I totally get that!

[Klemet]

Hey there !

Following @VLucet 's summon, I'd like to add that for the few things I've read so far, I think that this is a fantastic project !

It sure seems that a library such as DynamicGrids.jl could help models such as LANDIS-II run much faster than is currently the case. However, despite my experience in coding an extension for LANDIS-II, it's unclear to me if DynamicGrids.jl currently contains everything that's needed for LANDIS-II to be properly replicated.

LANDIS-II is not a very complex model under the hood. I'd say that it is very encapsulated, with some of its components being more complex than others (I think of the harvest libraries in particular). So, as long as equivalences are made between the main functions of LANDIS-II and functions in DynamicGrids.jl (or added functions in a potential Landscape.jl), I don't see why it would not be possible.

To me, the main problem lies in the fact that LANDIS-II already has an extensive and solid ecosystem of extensions that would have to be replicated in order for a hypothetical LANDIS.jl to be really useful, in my opinion. In addition, I think that such an affair should be done with the help of the current developers of LANDIS-II (@rmscheller and others), as their knowledge in ecology is an important part of LANDIS-II. The authors of the different existing extensions should also be contacted, as the maintenance and development of their extensions should be switched to Julia, maybe ?

But really, if such a project ever come to development phase, I'm of the opinion that it could be time to develop something that could be called LANDIS-III, since switching to Julia and being based on DynamicGrids.jl and others as I believe this would be a huge shift. Sadly, I really do not have the time to get invested in such a project right now. But it is very exciting !

[VLucet]

Thanks for your comment @Klemet.
Just to be clear, I did not intend to suggest replicating the LANDIS implementation with the rich diversity of associated extensions. I don't think the idea is a "LANDIS-III", but a separate library with similar principles and goals like others have done in R for example (I think LandR is an example). For what it's worth, I like the idea of a Landscape.jl package focused on performance and compatibility with other Julia packages, like Dispersal.jl and/or the EcoJulia group of packages.

[Klemet]

Thanks for the clarification, @VLucet : ). I'm sorry that I didn't picked up what you had in mind.

It sounds very interesting, then ! I don't think that this should be too hard to do. I only anticipate that several libraries will take a bit more time due to their complexity (again, thinking of the harvest libraries of LANDIS-II); but there rest should be quite easy to replicate in my opinion.

[rafaqz]

Hi @Klemet, good to hear from you here! we're excited to collaborate with LANDIS people. Knowledge of landscape change modeling is not something I or as far as I know anyone at cesar has currently, but we have a tool that could be useful for your field if it was coupled with enough domain specific knowledge.

I also didn't mean to suggest that Landscape.jl could replicate everything that LANDIS does, by any means! But some of the core functionality of LANDIS should be possible to implement with DynamicGrids.jl. Given that rules are easy to write in a script, early users can write custom components to fill the gaps. That kind of flexibility combined with performance/GPUs is the reason I imagine some may want to write models with DynamicGrids, not that it would match LANDIS on breadth of components.

it's unclear to me if DynamicGrids.jl currently contains everything that's needed for LANDIS-II to be properly replicated.

I also doubt it has everything. But it should get us a lot of the way to a basic first model. It would be good to try something simple and find out what's missing anyway. Then we can look at ease of writing models and the performance we get to think about whether it's worth investing more in the idea in future. If that sounds like its worth a shot?

@VLucet thank you for expanding this conversation! and to answer your questions:

Isn't this already possible with StaticArrays? Or maybe related to JuliaArrays/StaticArrays.jl#412?

StaticArrays.jl doesn't have a packed-bit implementation - so booleans need a whole word instead of a single bit. If I understand the model correctly (mostly from reading LANDIS Pro papers), we will want packed bits for the size of some LANDIS grids, so that e.g. 8*8 species/class matrix fits in a single UInt64, instead of 8 of them. Or a 4*4 in UInt16. It also means we can use bitwise operators on the whole matrix directly. This is where performance gains should happen, especially on GPUs. I'm collaborating with chelate from that issue to get this package released: https://github.com/chelate/StaticBitArrays.jl

However, my knowledge of (1) How LANDIS is implemented under the hood and (2) Package development in Julia remains very limited (for now!)

I wouldn't worry about this so much :) Package development in julia is easy compared to most other languages. If you can use git and write any Julia at all you can collaborate on a package. DynamiGrids rules are also easy to write, you are quite shielded from the internals. We probably can't use a lot of LANDIS implementation either, as we need to use a functional/GPU friendly style and multiple dispatch gives us different stuctural choices to OOP.

And it's really the specific ecology and math for the landscape-change domain that will be most important, and the use cases to apply. But DynamicGrids was written in a general enough way that these things should work, and it would be amazing if we could use models from multiple fields together in the future. So I would like to help this happen! I also expect to be writing any low-level parts that are required ;)

as I have use cases in mind that could be used as an example.

Sounds great. If you want to write out a description of the model we can chip away at implementing what it needs. That can form the bones of Landscape.jl

and/or the EcoJulia group of packages.

My idea was that Landscape.jl would be an EcoJulia package, if that sounds like a good idea?

Is this paper accessible? Would love to read it.

I'll look into putting it on BioArxiv. Re-reading it I remembered I unfortunately had to edit a lot of the landis discussion to cut 1000 words for submission. But it should give a good overview of the use cases.