Coupled without unnecessary functions

parents
# Compiled Object files
*.slo
*.lo
*.o
*.obj
# Precompiled Headers
*.gch
*.pch
# Compiled Dynamic libraries
*.so
*.dylib
*.dll
# Fortran module files
*.mod
# Compiled Static libraries
*.lai
*.la
*.a
*.lib
# Executables
*.exe
*.out
*.app
# Build directories
compiler-plugin/plugin_build/
runtime/build/
# Temporary files
*.swp
*log.txt
# Doxygen output
runtime/html/
Dhruva R. Chakrabarti (dhruva.chakrabarti@hpe.com)
Kumud Bhandari (kumud.bhandari@hpe.com)
Timothy McPhie (timothy.mcphie@hpe.com)
Kartik Singhal (kartik.singhal@hpe.com)
This diff is collapsed.
# Coupled-SFR
## Build instructions
Refer to build instructions in ./runtime/
## Persistent Region APIs
A preview is provided here. See `runtime/include/atlas_alloc.h` for the
actual interfaces.
A programmer needs to create one or more named persistent regions,
entities that hold everything persistent. The interface
**_NVM_FindOrCreateRegion_** or a variant can be used for this purpose. If a
region with the provided name exists, a handle to the region is
returned. Otherwise a region is created and its handle is
returned. Interfaces to close or delete a region are available.
To populate a persistent region, memory must be dynamically allocated
from that persistent region using **_nvm_alloc_** (or a variant) that has a
malloc-style interface. The region identifier must be provided so as
to identify the persistent region intended. An **_nvm_free_** is provided
for deallocation purposes.
Management of persistent regions and the contained data together
identify the persistent objects used by a program. Care must be taken
to ensure that all valid data within a persistent region is reachable
from the persistent root of the region. Use the interface
**_NVM_SetRegionRoot_** for this purpose.
## Consistency APIs
See `runtime/include/atlas_api.h` for the actual interfaces.
Persistent data must be kept consistent regardless of failures. The
programmer needs to call **_NVM_Initialize_** and **_NVM_Finalize_**
to start and stop Atlas support. Additionally, Atlas needs to know
code sections where program invariants are modified. If the program is
multithreaded and written using locks for synchronization, Atlas
automatically infers boundaries of regions where it must preserve
failure-atomicity (all-or-nothing) of updates to persistent
memory. Optionally, the programmer can demarcate sections of code with
calls to **_nvm_begin_durable_** and **_nvm_end_durable_** to identify
a durable or failure-atomic section of code. Note that no isolation
among threads is provided by a durable section. In contrast, if
persistent data is modified within a critical section, the critical
section provides both isolation among threads and durability to
persistent memory.
## Restart Code
A program might want to reuse data within a persistent region. For
this purpose, after finding a region handle, use the interface
**_NVM_GetRegionRoot_** to access the reachable data. Instead of
starting from scratch, this data can be reused to essentially restart
from where the region was left off the last time around.
That's all, as far as Atlas APIs are concerned. Compared to a
transient program, the idea is to write persistent memory programs
with as few changes as possible.
## Organization
- The APIs for this model are in `runtime/include`. [API doc here](http://hewlettpackard.github.io/Atlas/runtime/doc/atlas__api_8h.html).
- Instructions on how to build the compiler-plugin are in
`compiler-plugin/README.md`.
- Instructions on how to build the runtime are in `runtime/README.md`.
- For example programs using Atlas, see `runtime/tests`.
- The Atlas library sources are in `runtime/src`.
## Dependencies
* Currently, we support only x86-64 CPUs
* We assume Linux OS. Linux tmpfs must be supported. Testing has been
done on RedHat and Ubuntu.
* We assume modern C/C++ compilers in the build environment that must
support C/C++11.
* The default compiler used in the build is clang. Testing has been
done with version 3.6.0 or later. The instrumentation support is
currently available only with clang/LLVM. The runtime should build
with any compiler supporting C/C++11 though clang is preferred for
uniformity purposes.
* cmake version 3.1 or later
* boost library
* bash 4.0
For Ubuntu 16.04, these dependencies can be installed with:
sudo apt-get install llvm clang cmake libboost-graph-dev
* ruby (for certain test scripts), see **Installing Ruby** at [gorails](https://gorails.com/setup/ubuntu/16.04) for instructions.
[//]: # ( (c) Copyright 2016 Hewlett Packard Enterprise Development LP )
[//]: # ( )
[//]: # ( This program is free software: you can redistribute it and/or modify )
[//]: # ( it under the terms of the GNU Lesser General Public License as )
[//]: # ( published by the Free Software Foundation, either version 3 of the )
[//]: # ( License, or (at your option) any later version. This program is )
[//]: # ( distributed in the hope that it will be useful, but WITHOUT ANY )
[//]: # ( WARRANTY; without even the implied warranty of MERCHANTABILITY or )
[//]: # ( FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License )
[//]: # ( for more details. You should have received a copy of the GNU Lesser )
[//]: # ( General Public License along with this program. If not, see )
[//]: # ( <http://www.gnu.org/licenses/>. )
Compiler support for Atlas is currently in the form of an
instrumentation pass (NVM Instrumenter) built on top of the LLVM
compiler (http://llvm.org). Please ensure that LLVM/clang (version 3.6.0 or
above) is installed on your system or added to your PATH. If
required, visit http://llvm.org/releases/download.html to get the latest
binaries for this purpose.
To build the NVM Instrumenter shared library for clang, change
directory to `compiler-plugin` and run `./build_plugin`. The file
`NvmInstrumenter.so` will be created under the subdirectory
`plugin_build`.
Ensure the versions of clang and llvm developer tools (ie llvm-config) are the same.
To use clang with instrumentation, it is recommended to use an
environment variable as used in the Atlas test scripts
$ACC: clang -Xclang -load -Xclang <path_to_Atlas>/compiler-plugin/plugin_build/NvmInstrumenter.so
$ACXX: clang++ -Xclang -load -Xclang <path_to_Atlas>/compiler-plugin/plugin_build/NvmInstrumenter.so
#!/usr/bin/env bash
#
# (c) Copyright 2016 Hewlett Packard Enterprise Development LP
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Lesser General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version. This program is
# distributed in the hope that it will be useful, but WITHOUT ANY
# WARRANTY; without even the implied warranty of MERCHANTABILITY or
# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details. You should have received a copy of the GNU Lesser
# General Public License along with this program. If not, see
# <http://www.gnu.org/licenses/>.
#
logfile="build_log.txt"
if [ -f "$logfile" ]; then
rm $logfile
fi
srcfile="src/Instrumentation/NvmInstrumenter.cpp"
if [ "${1,,}" == "help" ];then
echo "Run with $0 to build NvmInstrumenter.so - the compiler plugin for building Atlas programs. Build commands are set to timeout after 5 minutes."
exit 0
fi
if [ ! -f "$srcfile" ]; then
echo "Could not find plugin source file NvmInstrumenter.cpp - are you running from within Atlas/compiler-plugin?"
exit 1
fi
clangpppath=$(which clang++; reval="$?")
if [ "$?" -ne 0 ]; then
echo "Could not find a copy of clang++, is it installed or added to PATH?"
exit 1
else
echo "Found clang++ in $clangpppath"
fi
llvmconfigpath=$(which llvm-config; reval="$?")
if [ "$?" -ne 0 ]; then
echo "Could not find llvm-config, is it installed or added to PATH?"
exit 1
else
echo "Found llvm-config in $llvmconfigpath"
fi
echo "Compiling object files" | tee $logfile
timeout 300s clang++ -c $srcfile `llvm-config --cxxflags` -fno-rtti >> $logfile 2>&1
retval="$?"
if [ "$retval" == "124" ]; then
echo "Compilation took longer than 5 minutes - have you got conflicting versions of llvmretval Try building with the linked script."
exit 1
elif [ "$retval" -ne 0 ]; then
echo "Build shared lib failed on compilation, check $logfile"
exit 1
else
echo "Compilation successful"
fi
echo "Linking" | tee $logfile
timeout 300s clang++ -shared NvmInstrumenter.o -o NvmInstrumenter.so -fno-rtti >> $logfile 2>&1
retval="$?"
if [ "$retval" == "124" ]; then
echo "Linking took longer than 5 minutes - have you got conflicting versions of llvmretval Try building with the linked script."
exit 1
elif [ "$retval" -ne 0 ]; then
echo "Build shared lib failed on linking, check $logfile"
exit 1
else
echo "Linking successful"
fi
rm NvmInstrumenter.o
if [ -f "NvmInstrumenter.so" ]; then
echo "Successfully built compiler plugin NvmInstrumenter.so"
rm $logfile
fi
if [ ! -d "plugin_build" ]; then
mkdir plugin_build
fi
mv NvmInstrumenter.so plugin_build/
echo "NvmInstrumenter.so is under plugin_build/"
This diff is collapsed.
[//]: # ( (c) Copyright 2016 Hewlett Packard Enterprise Development LP )
[//]: # ( )
[//]: # ( This program is free software: you can redistribute it and/or modify )
[//]: # ( it under the terms of the GNU Lesser General Public License as )
[//]: # ( published by the Free Software Foundation, either version 3 of the )
[//]: # ( License, or (at your option) any later version. This program is )
[//]: # ( distributed in the hope that it will be useful, but WITHOUT ANY )
[//]: # ( WARRANTY; without even the implied warranty of MERCHANTABILITY or )
[//]: # ( FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License )
[//]: # ( for more details. You should have received a copy of the GNU Lesser )
[//]: # ( General Public License along with this program. If not, see )
[//]: # ( <http://www.gnu.org/licenses/>. )
# Instrumentation Tests
`test_instr` performs instrumentation checks on different instructions, verifying
that the compiler plugin performs the correct instrumentations, both type and quantity.
## Usage
To run the test script do `./test_instr false`. Run with
`./test_instr true` to see debug information, if there are errors
for example.
## Requirements
The LLVM toolchain must be built debug with assertions enabled. This is needed in order to
run opt with -stats, more on the LLVM Statistic class
[here](http://llvm.org/docs/ProgrammersManual.html#statistic).
The compiler plugin `NvmInstrumenter.so` must also be built (with assertions).
If the compiler plugin is built outside of `plugin_build/`,
set environment variable PLUGIN to its location.
If these are not met the script will return with a non zero exit code.
## Adding New Tests
Both C or C++ tests are allowed.
Tests are placed in this directory `compiler-plugin/test`.
To add a new test `test.c` a corresponding `test.ref` must be created
and placed under `compiler-plugin/test/test_refs`.
A `.ref` file is used to check that the expected instrumentation
occured. These are generated with clang and opt. Run the following from this directory.
$ clang -c -emit-llvm test.c &> /dev/null
$ opt -load ../plugin_build/NvmInstrumenter.so -NvmInstrumenter -stats < test.bc > /dev/null 2> test.ref; rm test.bc
$ sed -i '/Number of/!d' test.ref
View the contents of `test.ref`. If the correct number of instrumentations have occurred,
move `test.ref` to `test_refs/`.
/*
* (c) Copyright 2016 Hewlett Packard Enterprise Development LP
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version. This program is
* distributed in the hope that it will be useful, but WITHOUT ANY
* WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
* for more details. You should have received a copy of the GNU Lesser
* General Public License along with this program. If not, see
* <http://www.gnu.org/licenses/>.
*/
void foo(long double *mloc)
{
*mloc = 1000;
}
/*
* (c) Copyright 2016 Hewlett Packard Enterprise Development LP
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version. This program is
* distributed in the hope that it will be useful, but WITHOUT ANY
* WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
* for more details. You should have received a copy of the GNU Lesser
* General Public License along with this program. If not, see
* <http://www.gnu.org/licenses/>.
*/
void foo(int *mloc)
{
*mloc = 1000;
}
/*
* (c) Copyright 2016 Hewlett Packard Enterprise Development LP
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version. This program is
* distributed in the hope that it will be useful, but WITHOUT ANY
* WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
* for more details. You should have received a copy of the GNU Lesser
* General Public License along with this program. If not, see
* <http://www.gnu.org/licenses/>.
*/
#include <stdio.h>
#include <stdlib.h>
#include <sys/time.h>
#include <string.h>
#include <unistd.h>
void foo( char *str1, char *str2){
strcpy(str2, str1);
strncpy(str2, str1, 5);
strcat(str2, str1);
strncat(str2, str1, 5);
}
#!/usr/bin/env bash
#
# (c) Copyright 2016 Hewlett Packard Enterprise Development LP
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Lesser General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version. This program is
# distributed in the hope that it will be useful, but WITHOUT ANY
# WARRANTY; without even the implied warranty of MERCHANTABILITY or
# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details. You should have received a copy of the GNU Lesser
# General Public License along with this program. If not, see
# <http://www.gnu.org/licenses/>.
#
debug_log=test_instr_log.txt
total_fails=0
out="instr_out.txt"
################################################################
# debug_print #
################################################################
function debug_print
{
local err_str="[TEST_INSTR] $1"
local echoon="$2"
if [ -z "$echoon" ]; then
echoon=$debug
elif [ "${echoon,,}" == "true" ]; then
echoon="true"
fi
if [ "$echoon" == "true" ]; then
echo "$err_str"
fi
echo "$err_str" >> $debug_log
}
################################################################
# failed_run #
################################################################
function failed_run
{
debug_print "Retaining .out .bc files for debugging purposes" "true"
total_fails=$(($total_fails+1))
}
################################################################
# test_instrumentation #
################################################################
function test_instrumentation
{
debug_print "Beginning instrumentation testing" "true"
test_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
cd $test_dir
plugin="$test_dir/../plugin_build/NvmInstrumenter.so"
atlas_root="$test_dir/../.."
if [ -f "$plugin" ]; then
debug_print "Found compiler plugin build"
elif [ ! -z "$PLUGIN" ]; then
debug_print "Found compiler plugin to test instrumentation in $PLUGIN"
plugin="$PLUGIN"
else
debug_print "Cannot detect a build of atlas instrumentation plugin, ensure it is built with compiler-plugin/build_plugin." "true"
debug_print "If your build plugin is outside of compiler_plugin, set \$PLUGIN to it's location" "true"
exit 1
fi
if ! which opt > /dev/null ; then
debug_print "Cannot find opt - cannot conduct instrumentation testing, exiting" "true"
exit 1
fi
if opt --version | grep assert > /dev/null; then
debug_print "Found a version of opt built with asserts"
else
debug_print "No version of opt with asserts found, cannot check statistics, exiting" "true"
exit 1
fi
tests=($(ls -d *.c *.cpp 2> /dev/null))
for testname in ${tests[@]}; do
testname=${testname%.*}
if [ ! -f test_refs/$testname\.ref ]; then
debug_print "Test $testname does not have a .ref file under test_refs. If this test has just been added, consult the README on how to generate a corresponding .ref file" "true"
debug_print "Counting as a failure, running next test" "true"
failed_run
continue
fi
debug_print "Generating llvm ir bitcode file for $testname"
clang -c -emit-llvm $testname\.c -I$atlas_root/runtime/include &> /dev/null
if [ "$?" -ne 0 ]; then
debug_print "Failed to generate bitcode for $testname, running next test" "true"
failed_run
continue
fi
debug_print "Using opt to get instrumentation readout of bitcode for $testname"
opt -load $plugin -NvmInstrumenter -stats < $testname\.bc > /dev/null 2> $testname\.out
if [ "$?" -ne 0 ]; then
debug_print "Failed to run opt for $testname, running next test" "true"
failed_run
continue
fi
debug_print "Using sed to sanitise $testname.out"
sed -i '/Number of/!d' $testname\.out
debug_print "Results of $testname are" "true"
cat $testname\.out | tee -a $debug_log
debug_print "Running diff, output is in $debug_log"
if diff $testname\.out test_refs/$testname\.ref >> $debug_log; then
debug_print "Instrumentation results were as expected for $testname" "true"
debug_print "Run successful - removing $testname\.out and $testname\.bc"
rm -f $testname\.out $testname\.bc
else
debug_print "Instrumentation stats did not match expected values for $testname" "true"
debug_print "Counting as a failure - view diff in $debug_log" "true"
failed_run
fi
done
}
################################################################
# main #
################################################################
#Run with debug flag set to true to see output of build commands and which tests are passing.
#which would have been displayed with debug flag set to true. test_region_log contains output of tests.
help_str="USAGE: ./test_instr [debug flag: true or false] \nVerifies correctness of instrumentation by compiler_plugin. \nRequires a debug build of LLVM with assertions and the compiler plugin to be built."
debug="$1"
if [ "${debug,,}" == "false" ]; then #bash4.0 convert to lower
debug="false"
elif [ "${debug,,}" == "true" ]; then #bash4.0 convert to lower
debug="true"
else
echo -e "$help_str"
exit 1
fi
rm -f $debug_log
debug_print "Set debug value to $debug"
test_instrumentation
if [[ $total_fails -gt 0 ]]; then
debug_print "$total_fails fails for region tests" "true"
exit 1
else
debug_print "No failures occurred, instrumentation testing passed" "true"
exit 0
fi
1 nvm_instr - Number of stores instrumented
1 nvm_instr - Number of stores instrumented
2 nvm_instr - Number of strcats instrumented
2 nvm_instr - Number of strcpys instrumented
This diff is collapsed.
## Build instructions
$ cd runtime
$ mkdir build-all
$ cd build-all
$ cmake .. -DFLC=true -DNO_NES