Hello, I would like to run a simulation using the host and MIC. In a recent question in this forum I received a large amount of help from the Intel team (thank you, intel) . I was shown that a SOA data type format was most advantageous for me. The issue with this for me here, is that the arrays in this structure should be allocatable in my code. Thus, I can't transfer them from the host to the MIC. On the host, my algorithm has a AOS type with only doubles and integers in the data type, and hence is bit wise copyable and should be able to be transferred to the MIC. Thus, I would like to copy the AOS array to the MIC, and then move the AOS data to the SOA variable. I also have another data type that is constant, but whose values are determined at run time. I would like to have this variable on both the host and MIC. However, I have no idea how to declare variables solely on the MIC. Also one of the SOA variables, needs to be deallocated and reallocated on MIC if a conditions is met. In the code below, I have written my AOS array as a normal array for simplicity.
1. read in variable from file (np). Allocate my AOS variable on Host. Allocate SOA variable on MIC
2. read in variable from file (cellT). Calculate list on host and MIC
3. Transfer AOS data to MIC, if necessary, deallocate and reallocate tr%start and tr%end
I have written the following code showing the procedure with ample comments to hopefully make it clear. All of the action is in the subroutine offload
program vec
!--- all global variables are in modcell_en below
use ifport
use modcell_en
implicit none
!--- convention array. This would be my AOS in my actual code
double precision,allocatable :: rnew(:,:)
integer,allocatable :: start(:), fin(:)
!--- bs variables
double precision :: energy
double precision :: x1,y1,z1,x2,y2,z2,dr2
integer :: i,j,k,l,count,np,cellT
integer :: c1s,c1e,c2s,c2e
!--- in my real code, this variable is readin from a file
!--- represents the total number of atoms in a simulation
np = 60000
allocate(rnew(3,np)
!--- in my real code, this is also read in from file
cellT = 1000
list%ncell = cellT
allocate(start(cellT),fin(cellT))
!--- allocate arrays on MIC
call offload(0)
call srand(10)
!--- dont care about this. this is just initializing random positions
do i = 1,np
rnew(1,i) = 10*rand(); rnew(2,i) = 10*rand(); rnew(3,i) = 10*rand()
end do
!--- don't care about this either. just assigning pointers to rnew and part
count = 0
do i = 1,cellT
start(i) = count*60+1
fin(i) = (count+1)*60
count = count + 1
enddo
call get_energy(energy)
stop
end program vec
module modcell_en
!--- global structure of array position variable
!--- this variable will only be used on the MIC
!--- Hence, I would like to declare and allocate it on the MIC
!--- data must be passed to the mic however, to populate this data
type r
double precision,allocatable :: x(:),y(:),z(:)
end type r
!--- global structure of array variable
type(r) :: part
!--- This variable will also only be used on the MIC
!--- This variable may need to be deallocated and reallocated during the simulation
!--- information must be passed to the MIC to populate this varaible
type tr
integer,allocatable ::start(:),fin(:)
end type tr
!--- number of cells in simulation
!--- this variable will be used both on host and MIC BUT IS CONSTANT
!--- would like to transfer once and have it stay there forever
type list
integer :: ncell
end type list
contains
subroutine offload(check)
integer :: check
integer :: i,j
!--- simulation just started. allocate space on MIC for variables
!--- should allocate part and tr here, but dont know how
!--- should transfer list here, and have it stay there
if(check.eq.0)then
!dir$ offload_transfer target(mic:0) in(r:alloc_if(.true.) free_if(.false.)),&
!dir$ in(list:alloc_if(.true) free_if(.false.)),&
!dir$ in(start:alloc_if(.true.) free_if(.false.)),&
!dir$ in(fin:alloc_if(.true.) free_if(.false.))
endif
if(check.eq.1)then
!--- need to transfer data from variable on host to variable on MIC
!--- np is large in my simulation (~60,000-70,000), so the MIC should vectorize and transfer this easily I hope
!dir$ offload_transfer target(mic:0) nocopy(r:alloc_if(.false.) free_if(.false.)),&
!dir$ nocopy(start:alloc_if(.false.) free_if(.false.)),&
!dir$ nocopy(fin:alloc_if(.false.) free_if(.false.))
!$omp parallel do
do i = 1,np
r%x(i) = r(1,i)
r%y(i) = r(2,i)
r%z(i) = r(3,i)
end do
do i = 0,list%cellT
tr%start(i) = start(i)
tr%fin(i) = fin(i)
enddo !$omp end parallel do
!dir$ end offload
endif
!--- number of cells in simulation has changed
!--- need to deallocate and reallocate tr%start and tr%end
!--- strill need to transfer r
if(check.eq.2)then
!dir$ offload begin target(mic:0) in(list:alloc_if(.true) free_if(.false.)),&
!dir$ in(start:alloc_if(.true.) free_if(.false.)),&
!dir$ in(fin:alloc_if(.true.) free_if(.false.))
!dir$ nocopy(r:alloc_if(.false.) free_if(.false.))
!$omp parallel do
do i = 1,np
r%x(i) = r(1,i)
r%y(i) = r(2,i)
r%z(i) = r(3,i)
end do
do i = 0,list%cellT
tr%start(i) = start(i)
tr%fin(i) = fin(i)
enddo
!$omp end parallel do
!dir$ end offload
endif
end subroutine offload
subroutine get_energy(energy)
implicit none
double precision :: energy, pot
double precision :: x1,y1,z1
double precision :: x2,y2,z2
double precision :: dr,dr2,dri,dr2i,dr6i,dr12i
integer :: i,j,k,l
integer :: c1s,c1e,c2s,c2e
integer :: check
if(condition.eq.1)then
call offload(1)
elseif(condition.eq.2)then
call offload(2)
endif
!dir$ offload begin target(mic:0)
!$omp parallel do reduction(+:energy)
!---main code I would like to port to MIC in array of structure format
do i= 1,list%ncell
c1s = tr%start(i); c1e = tr%fin(i)
do j = 1,ncell
c2s = tr%start(j); c2e = tr%fin(j)
do l=c2s,c2e
x1 = part%x(l); y1 = part%y(l); z1 = part%z(l)
do k = c1s,c1e
x2 = part%x(l); y2 = part%y(l); z2 = part%z(l)
dr2 = (x2-x1)**2+(y2-y1)**2+(z2-z1)**2
if(dr2.lt.2)then
dr = sqrt(dr2)
dri = 1.0d0/dr
dr2i = dri*dri
dr6i = dr2i**3
dr12i = dr6i**2
energy = energy + 4.0d0*(dr12i-dr6i)
endif
enddo
enddo
enddo
enddo
!$omp end parallel do
!dir$ end offload
end subroutine get_energy
end module modcell_en