Generating PVD Files When The Triangulation Is Not Present On All Ranks

by ADMIN 72 views

Introduction

In the current implementation of the PVD file generation, a major limitation exists when the triangulations are not present on all MPI ranks. This issue has been documented in issue #70 as a #TODO task for taking nothing. However, for users who require this functionality, it is essential to address this limitation. In this article, we will explore the changes made to overcome this issue and provide a detailed explanation of the modified code.

Problem Statement

The current implementation of the PVD file generation relies heavily on the presence of triangulations on all MPI ranks. However, in certain scenarios, this may not be the case, leading to an inability to generate PVD files. This limitation is a significant roadblock for users who require this functionality.

Solution Overview

To address this limitation, we need to modify the existing code to handle scenarios where triangulations are not present on all MPI ranks. This involves introducing a new data structure, DistributedPvd, to store the PVD files and their corresponding main ranks. We will also modify the existing functions, createpvd and savepvd, to accommodate this new data structure.

Modified Code

The modified code introduces a new struct, DistributedPvd, to store the PVD files and their corresponding main ranks.

struct DistributedPvd{T<:AbstractArray}
  pvds::T
  main_rank::Int
end

The createpvd function is modified to handle scenarios where triangulations are not present on all MPI ranks.

function Visualization.createpvd(trian::DistributedTriangulation,parts::AbstractArray,args...;kwargs...)
  main_rank = -1
  map(local_views(trian), parts) do LocTrian, part
    flag = part * !(num_cells(LocTrian) == 0)
    global_flag = Vector{Int}(undef, MPI.Comm_size(MPI.COMM_WORLD))
    MPI.Allgather!([flag], global_flag, MPI.COMM_WORLD)
    main_rank = maximum(global_flag)
  end
  pvds = map_main(parts,main = main_rank) do part
    p = paraview_collection(args...;kwargs...)
  end
  DistributedPvd(pvds,main_rank)
end

The createpvd function is also modified to handle scenarios where a function f is provided to process the PVD files.

function Visualization.createpvd(f,trian::DistributedTriangulation,parts::AbstractArray,args...;kwargs...)
  pvd = createpvd(trian,parts,args...;kwargs...)
  try
    f(pvd)
  finally
    savepvd(pvd)
  end
end

The savepvd function is modified to handle the new DistributedPvd data structure.

function Visualization.savepvd(pvd::DistributedPvd)
  map_main(pvd.pvds, main = pvd.main_rank) do pvd
    vtk_save(pvd)
  end
end

The setindex! function is modified to handle the new DistributedPvd data structure.

function Base.setindex!(pvd::DistributedPvd,pvtk::AbstractArray::Real)

  map(pvtk) do pvtk
    if !isnothing(pvtk)
      vtk_save(pvtk)
    end  
  end

  map_main(pvtk,pvd.pvds,main = pvd.main_rank) do pvtk,pvd
    pvd[time] = pvtk
  end
end

Conclusion

In this article, we have explored the changes made to overcome the limitation of generating PVD files when triangulations are not present on all MPI ranks. The modified code introduces a new data structure, DistributedPvd, to store the PVD files and their corresponding main ranks. We have also modified the existing functions, createpvd and savepvd, to accommodate this new data structure. The modified code provides a robust solution for generating PVD files in scenarios where triangulations are not present on all MPI ranks.

Future Work

Future work includes further optimization of the modified code to improve performance and scalability. Additionally, exploring other use cases where the modified code can be applied to provide value to users.

References

Introduction

In our previous article, we explored the changes made to overcome the limitation of generating PVD files when triangulations are not present on all MPI ranks. In this article, we will address some of the frequently asked questions (FAQs) related to this topic.

Q: What is the main limitation of the current implementation of PVD file generation?

A: The current implementation of PVD file generation relies heavily on the presence of triangulations on all MPI ranks. However, in certain scenarios, this may not be the case, leading to an inability to generate PVD files.

Q: What is the new data structure introduced in the modified code?

A: The new data structure introduced in the modified code is called DistributedPvd. It stores the PVD files and their corresponding main ranks.

Q: How does the createpvd function handle scenarios where triangulations are not present on all MPI ranks?

A: The createpvd function uses the map function to iterate over the local views of the triangulation and the parts. It then uses the MPI.Allgather! function to gather the flags indicating whether the triangulation is present on each rank. The main rank is determined by finding the maximum flag value.

Q: How does the savepvd function handle the new DistributedPvd data structure?

A: The savepvd function uses the map_main function to iterate over the PVD files and their corresponding main ranks. It then uses the vtk_save function to save each PVD file.

Q: What is the purpose of the setindex! function in the modified code?

A: The setindex! function is used to set the value of a PVD file at a specific time. It uses the map function to iterate over the PVD files and their corresponding main ranks, and then uses the map_main function to set the value of each PVD file.

Q: What are the benefits of the modified code?

A: The modified code provides a robust solution for generating PVD files in scenarios where triangulations are not present on all MPI ranks. It also improves the performance and scalability of the code.

Q: What are the future work directions for the modified code?

A: Future work includes further optimization of the modified code to improve performance and scalability. Additionally, exploring other use cases where the modified code can be applied to provide value to users.

Q: What are the references for the modified code?

A: The references for the modified code include the issue #70, Mpi.jl, and Paraview.

Conclusion

In this article, we have addressed some of the frequently asked questions (FAQs) related to the modified code for generating PVD files when triangulations are not present on all MPI ranks. We hope that this Q&A article has provided valuable insights and information to users who are interested in this topic.

References