Hi,
I'm having trouble running some offload code. The code compiled fine and appeared to execute until it needed some functions that were defined as MIC only and threw the following error:
[Offload] [HOST] [State] Initialize logical card 0 = physical card 0
On the remote process, dlopen() failed. The error message sent back from the sink is /var/volatile/tmp/coi_procs/1/7016/load_lib/iccoutyjfaoE: undefined symbol: bfgsfit_visibilities_mic
offload error: cannot load library to the device 0 (error code 20)
On the sink, dlopen() returned NULL. The result of dlerror() is "/var/volatile/tmp/coi_procs/1/7016/load_lib/iccoutyjfaoE: undefined symbol: bfgsfit_visibilities_mic"
[Offload] [HOST] [State] Unregister data tables
I read about some similar issues on the forum and it seems to me that there is some kind of linking problem. A library made with xiar (and containing the MIC code) should be linked by the following makefile:
OUTPUT=
CXX=icpc
$MKLROOT=/opt/intel/composer_xe_2013.5.192/mkl
IFACE_LIB=mkl_intel_lp64
THREADING_LIB=mkl_intel_thread
CORE_LIB=mkl_core
LDFLAGS=-L$(MKLROOT)/lib/intel64 -l$(IFACE_LIB) -l$(THREADING_LIB) -l$(CORE_LIB) -lpthread
MIC_LDFLAGS=-L$(MKLROOT)/lib/mic -l$(IFACE_LIB) -l$(THREADING_LIB) -l$(CORE_LIB) -L$(MIC_LD_LIBRARY_PATH)
#MICFLAGS =-offload-option,mic,compiler,"-DUSE_MIC -vec-report1 -g -O0 -Wall"
MICFLAGS =-offload-option,mic,compiler,"-DUSE_MIC -vec-report1 -O3 -openmp -Wall"
#MICFLAGS =-offload-option,mic,compiler,"-DUSE_MIC -O1 -profile-functions -profile-loops=all -profile-loops-report=2 -openmp"
MICLDFLAGS=-offload-option,mic,ld,"$(MIC_LDFLAGS)"
CXXFLAGS +=-DUSE_MIC -g -O3 -Wall -openmp -vec-report=1 $(MICFLAGS)
#CXXFLAGS +=-DUSE_MIC -O1 -profile-functions -profile-loops=all -profile-loops-report=2 -openmp -vec-report=1 $(MICFLAGS)
CASA_LIBDIR=/home/coughlan/phi_stuff/lofar_software_install/lib
CASA_INCDIR=/home/coughlan/phi_stuff/lofar_software_install/include/casacore
CASA_LIBS=-lcasa_casa -lcasa_tables -lcasa_measures -lcasa_ms -lcfitsio
#LDFLAGS=-Wl,-t,--rpath,/software/users/lofareor/SW/lib64
# -Wl,--hash-style=both
# with multithread FFTW
MY_LIBS=-lsagecal
INCLUDES=-I. -I./lib -I$(CASA_INCDIR) -I/usr/include
LIBPATH=-L$(CASA_LIBDIR) -L./lib
#### glib
GLIBI=-I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/lib64/glib-2.0/include/
GLIBL=-lglib-2.0
OBJECTS=main.o data.o
default:sagecal
main.o:main.cpp
$(CXX) $(CXXFLAGS) $(INCLUDES) $(GLIBI) -c $<
data.o:data.cpp data.h
$(CXX) $(CXXFLAGS) $(INCLUDES) $(GLIBI) -c $<
sagecal:$(OBJECTS) ./lib/libsagecal.a
$(CXX) $(CXXFLAGS) $(LDFLAGS) $(MICFLAGS) $(MICLDFLAGS) $(INCLUDES) $(GLIBI) $(LIBPATH) -o $@ $(OBJECTS) $(MY_LIBS) $(CASA_LIBS) $(GLIBL)
clean:
rm *.o *.tmp *.fits
When I look at the library with nm I get
bash-4.1$ nm -C ./lib/libsagecal.a| grep vis
0000000000002080 T predict_visibilities
0000000000003fc0 T bfgsfit_visibilities
0000000000005e30 T bfgsfit_visibilities_mic
00000000000026c0 T sagefit_visibilities
00000000000047a0 T sagefit_visibilities_mic
0000000000005220 T predict_visibilities_multifreq
0000000000004a50 t visibilities_threadfn_multifreq
bash-4.1$ nm -C sagecal| grep vis
000000000043b1e0 T predict_visibilities
0000000000443110 T predict_visibilities_multifreq
0000000000442940 t visibilities_threadfn_multifreq
The function mentioned in the error (bfgsfit_visibilities_mic) is missing from the final executable. Is this the source of the problem (or is the compiled code somehow located elsewhere like the MIC.o files). If not, could you give me some pointers as to what might be the issue?
Thanks for your help,
Colm