Versions Compared

Version Old Version 9 New Version Current
Changes made by

Hiren Vadalia

Hiren Vadalia

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


OSS Site Map

Developer Guide Pages


This page shows how to run automated tests while PBS is running under memory diagnostics tools like valgrind, electric fence or purify

However, this page mostly focus focuses on valgrind Valgrind only but this step can be used to run any diagnostics tools such as code coverage tools, etc...

Step-by-step guide

Compile Python with

valgrind

Valgrind support

  1. Install valgrind devel package
    1. Command: yum install valgrind-devel
  2. Compile python from source as follow

    1. ./configure --prefix=<python_install_dir> --without-pymalloc --with-pydebug --with-valgrind

    2. make

    3. make install

Compile and install PBS with above python and in partial debug mode

  1. ./configure --prefix=<installdir> --with-python=<python_install_dir> CFLAGS="-g -O0 -DPy_DEBUG -Wall -Werror"
    1. NOTE: Do not add '-DDEBUG' option in CFLAGS because it will start all PBS daemons in foreground mode which will hang PBS init script forever
  2. make; make install

Change in pbs_init.d script

NOTE: This step applies only if you are using PBS code before https://github.com/PBSPro/pbspro/commit/d46352b21eb76c38a21849b987b0b9dd719a839a commit

Replace below lines in check_started() as shown


Code Block
languagebash
# strip out everything except executable name
  prog_name=`echo ${ps_out} | awk '{print $1}' | \
       awk -F/ '{print $NF}' | awk '{print $1}'`
  if [ "${prog_name}" = $2 ] ; then


with

Code Block
languagebash
 # strip out everything except executable name
   prog_name=`echo ${ps_out} | grep -how "$2"`
   if [ "x${prog_name}" = "x$2" ] ; then

Generate diagnostics tool wrapper script for binaries

Code Block
languagebash
#!/bin/bash

LOG_DIR=
BASE_SUPP=
PBS_CONF_FILE=${PBS_CONF_FILE:-/etc/pbs.conf}

generate_valgrind_wrapper() {
  srcbin="${1}"
  binname="$(basename ${srcbin})"
  destbin="${srcbin}.vlgd"
  if [ -x "${srcbin}" ]; then
    __val_opts='--tool=memcheck --leak-check=full --track-origins=no'
    __val_opts="${__val_opts} --child-silent-after-fork=yes"
    __val_opts="${__val_opts} --redzone-size=256 --track-fds=yes"
    __val_opts="${__val_opts} --log-file=${LOG_DIR}/${binname}_\$(date \"+%s\").log"
    __val_opts="${__val_opts} --vgdb=no --xml=no"
    if [ "x${BASE_SUPP}" != "x" -a -f "${BASE_SUPP}" ]; then
      __val_opts="${__val_opts} --suppressions=${BASE_SUPP}"
    fi
    __wait_time=5
    if [ "X${binname}X" == "Xpbs_server.binX" ]; then
      __wait_time=$((__wait_time * 4))
    fi
    if [ ! -x "${destbin}" ]; then
      mv "${srcbin}" "${destbin}"
      if [ $? -ne 0 ]; then
        echo "Failed to copy \"${srcbin}\" to \"${destbin}\""
        exit 1
      fi
    fi
    {
      echo '#!/bin/bash'
      echo 'if [ "$1" == "--version" ]; then'
      echo "  ${destbin} --version"
      echo 'else'
      echo "  valgrind ${__val_opts} ${destbin} -N \"\$@\" &"
      if [ ${__wait_time} -ne 0 ]; then
        echo "  sleep ${__wait_time}"
      fi
      echo 'fi'
    } > /tmp/valwrapper
    mv /tmp/valwrapper "${srcbin}"
    if [ $? -ne 0 ]; then
      echo "Failed to copy valwrapper to \"${srcbin}\""
      exit 1
    else
      echo "*** Generated valgrind wrapper for ${srcbin}"
      echo "========================================"
      cat "${srcbin}"
      echo "========================================"
    fi
    chmod +x "${srcbin}"
  fi
}

usage() {
  echo "$0 --log-dir <path to valgrind log dir> --base-supp <base suppression file to use>"
}

parse_params() {
  while [ $# -gt 0 ]
  do
    case $1 in
      --log-dir) LOG_DIR="$2"; shift 2;;
      --base-supp) BASE_SUPP="$2"; shift 2;;
      *) echo "Invalid option: $1"; echo; usage; exit 1;;
    esac
  done
}

parse_params "$@"
if [ "x$(id -u)" != "x0" ]; then
	echo "This script must be run by root!"
	exit 1
fi
if [ "x${LOG_DIR}" == "x" ]; then
  usage
  exit 1
fi
if [ ! -f ${PBS_CONF_FILE} ]; then
	echo "Couldn't find pbs.conf!"
	exit 1
fi

. "${PBS_CONF_FILE}"

if [ "x${PBS_EXEC}" == "x" ]; then
	echo "Invalid PBS_EXEC in pbs.conf!"
	exit 1
fi
${PBS_EXEC}/libexec/pbs_init.d stop
mkdir -p "${LOG_DIR}"
generate_valgrind_wrapper "${PBS_EXEC}/sbin/pbs_server.bin"
generate_valgrind_wrapper "${PBS_EXEC}/sbin/pbs_mom"
generate_valgrind_wrapper "${PBS_EXEC}/sbin/pbs_sched"
generate_valgrind_wrapper "${PBS_EXEC}/sbin/pbs_comm"
${PBS_EXEC}/libexec/pbs_init.d start


NOTES:

  • The above script only generates wrapper script for all PBS daemons only, but you can modify above script for any binaries which you want to run under diagnostics tool with any options
  • The option --vgdb is not supported in Valgrind versions below 3.7.0. Systems with RHEL 6 have valgrind version below 3.7.0.


Now run your automated test.

OSS Site Map

Developer Guide Pages