dead1e c0ffee

On AIX 6.1, we see the following error:

# oslevel -s
rpm_share: 0645-024 Unable to access directory /tmp/.workdir.487456.430278_1
rpm_share: 0645-007 ATTENTION: init_baselib() returned an unexpected result.
6100-02-05-0939
#

This may be something we can further debug, if oslevel is a shell script.

First of all, figure out where oslevel is:

# whereis oslevel
oslevel: /usr/bin/oslevel

Next, figure out what it is:

# file /usr/bin/oslevel
/usr/bin/oslevel: shell script  - ksh (Korn shell)

Now, that we know its a shell, script, we put in a set -x. To do this to the beginning, you
can just put a -x after the ksh line:

Change:

#!/bin/ksh
# IBM_PROLOG_BEGIN_TAG
# This is an automatically generated prolog.
#
# bos53L src/bos/usr/bin/oslevel/oslevel.sh 1.5.6.2
#

To this:

#!/bin/ksh -x
# IBM_PROLOG_BEGIN_TAG
# This is an automatically generated prolog.
#
# bos53L src/bos/usr/bin/oslevel/oslevel.sh 1.5.6.2
#

Then, run it again (I only show the end of the command):

+ [[ 1 = 1 ]]
+ [[ 0 = 1 ]]
+ [[ 0 = 1 ]]
+ [[ 0 = 1 ]]
+ print_current_spack
rpm_share: 0645-024 Unable to access directory /tmp/.workdir.667776.684286_1
rpm_share: 0645-007 ATTENTION: init_baselib() returned an unexpected result.
6100-02-05-0939
+ exit 0
+ interrupted

Looks like we are looking for a function called ‘print_current_spack’, the set -x doesn’t descend
into functions, so take the original one out and add one into this function:

Before:

print_current_spack()
{
        trap interrupted INT QUIT TERM
        typeset BUF
        typeset DVRMF
        typeset DSPNO
        typeset DTLNO

After:

print_current_spack()
{
        set -x
        trap interrupted INT QUIT TERM
        typeset BUF
        typeset DVRMF
        typeset DSPNO
        typeset DTLNO

Now we see the next level of debugging show up at the top of the output:

# oslevel -s | more
+ trap interrupted INT QUIT TERM
+ typeset BUF
+ typeset DVRMF
+ typeset DSPNO
+ typeset DTLNO
+ get_known_spacks
rpm_share: 0645-024 Unable to access directory /tmp/.workdir.536746.573476_1
rpm_share: 0645-007 ATTENTION: init_baselib() returned an unexpected result.
+ + /bin/grep _SP: /tmp/oslevel.0.577656/.oslevel.mlinfo
+ /bin/grep :-:
+ /usr/bin/awk -F: { print $1 }

So we just move on to ‘get_known_spacks’. In this case, I can’t find the output easily on the screen,
so I write to a file:

oslevel -s > /tmp/outfile 2>&1

print_current_rml (this may now be official spaghetti code):

+ typeset MVRMF
+ typeset SPNO
+ typeset TLNO
+ + print_current_rml
rpm_share: 0645-024 Unable to access directory /tmp/.workdir.585812.594022_1
rpm_share: 0645-007 ATTENTION: init_baselib() returned an unexpected result.
BUF=6100-02-00_SP
+ [[ ! -s /tmp/oslevel.0.684192/.oslevel.mlinfo ]]
+ + /usr/bin/awk -F: $1 ~ /_SP$/ { print $1 } /tmp/oslevel.0.684192/.oslevel.mlinfo
+ /usr/bin/sort -t- -r
+ /usr/bin/uniq

Next function = get_known_rmls:

+ trap interrupted INT QUIT TERM
+ get_known_rmls
rpm_share: 0645-024 Unable to access directory /tmp/.workdir.557148.569454_1
rpm_share: 0645-007 ATTENTION: init_baselib() returned an unexpected result.
+ + /bin/grep _AIX_ML /tmp/oslevel.0.340030/.oslevel.mlinfo
+ grep :-:
+ /usr/bin/awk -F: { print $1 }
+ /usr/bin/uniq

print_min_rml:

+ trap interrupted INT QUIT TERM
+ typeset BUF
+ + print_min_rml
rpm_share: 0645-024 Unable to access directory /tmp/.workdir.512036.557160_1
rpm_share: 0645-007 ATTENTION: init_baselib() returned an unexpected result.
BUF=6120-00
+ [[ ! -s /tmp/oslevel.0.553194/.oslevel.mlinfo ]]
+ + /usr/bin/awk -F: $1 ~ /_AIX_ML$/ { print $1 } /tmp/oslevel.0.553194/.oslevel.mlinfo
+ /usr/bin/uniq
+ /usr/bin/sort -r
KWN_RML_LEVS=6100-02_AIX_ML
6100-01_AIX_ML
6100-00_AIX_ML
5300-08_AIX_ML

Now, it might be getting trickier:

+ trap interrupted INT QUIT TERM
+ typeset BUF
+ + print_min_rml
+ typeset ERROR=eval /usr/sbin/inuumsg 216 110 >&2; exit 1
+ typeset BUF
+ [[ 1 -ne 1 ]]
+ + /usr/bin/lslpp -qLc bos.rte
+ LC_ALL=C
rpm_share: 0645-024 Unable to access directory /tmp/.workdir.377012.446524_1
rpm_share: 0645-007 ATTENTION: init_baselib() returned an unexpected result.
BUF=bos:bos.rte:6.1.2.1: : :C:F:Base Operating System Runtime: : : : : : :0:0:/:0920
+ + echo bos:bos.rte:6.1.2.1: : :C:F:Base Operating System Runtime: : : : : : :0:0:/:0920
+ awk -F: {split($3,lev,".");
         print lev[1]lev[2]lev[3]"0"; exit(0)}
+ LC_ALL=C

Now, it gets more difficult. First we see that rpm_share is not something called directly, although some research shows that it is a different script. Set -x in it gets stripped out, so that doesn’t help.

Here is the source:

print_min_rml()
{
   typeset ERROR='eval /usr/sbin/inuumsg 216 110 >&2; exit 1'
   typeset BUF

   [[ $recml -ne 1 ]] && return 0

   # There is no RML information, lets print out
   # VRMF-00.
   BUF=$(LC_ALL=C /usr/bin/lslpp -qLc bos.rte) || $ERROR
   BUF=$(echo "$BUF" | LC_ALL=C awk -F: '{split($3,lev,".");
         print lev[1]lev[2]lev[3]"0"; exit(0)}')
   [[ $? -ne 0 || -z $BUF ]] && $ERROR
   echo "${BUF}-00"
   return 0

}  # end of print_min_rml()

Lets walks though it from the shell, one line at a time:

# BUF=$(LC_ALL=C /usr/bin/lslpp -qLc bos.rte)
# echo $BUF
bos:bos.rte:6.1.2.1: : :C:F:Base Operating System Runtime: : : : : : :0:0:/:

I include the echo to make sure we got something, so we move onto the next line:

 echo "$BUF" |  LC_ALL=C awk -F: '{split($3,lev,"."); print lev[1]lev[2]lev[3]"0"; exit(0)}'
6121

I would have done that mass off ugliness differently, but now we have BUF=6121

We are also at the end of this function, it is supposed to return 6.1.2.0-00, and does so, but also throws an error. Lets try to hard-code something to isolate it. This means stepping back a function:

get_known_rmls()
{
   trap interrupted INT QUIT TERM
   typeset BUF

#   BUF=$(print_min_rml)
        BUF="6.1.2.0-00"

   if [[ ! -s $mlinfo ]]; then
      KWN_RML_LEVS=$BUF
      return 0

This works fine:

# oslevel -s
6100-02-05-0939

So know we know the problem must be in ‘print_min_rml’. When we removed the hard-coding, it still seemed to work, maybe it cleared out an unknown wierdness.