www.openlinksw.com
docs.openlinksw.com

Book Home

Contents
Preface

Installation Guide

Virtuoso for Windows
Virtuoso for Linux (Enterprise Edition)
Virtuoso for Unix (Enterprise Edition)
Virtuoso for Unix (Personal Edition)
Virtuoso for Mac OS X
Virtuoso ADO.Net Data Grid Form Application
Using Visual Studio 2008 to Build an Entity Frameworks based Windows Form Application
Using Visual Studio 2008 to Build an ADO.NET Data Services based Application
Windows Form Application for accessing Virtuoso RDF data via SPASQL using the Virtuoso ADO.Net Provider
Creating a Web Browser Application to Access RDF Data Using The Virtuoso ADO.Net Provider
Creating a Silverlight Application to consume the service
Creating A Simple .NET RIA Services Application To Display Data From Virtuoso
Creating a .Net RIA Services Application That Will Update Virtuoso Data
Cluster Installation and Config
Creating New Cluster Setup Cluster Config

2.14. Cluster Installation and Config

2.14.1. Creating New Cluster Setup

After the installation of V6 is done, in order to setup a Cluster you need to perform the following steps (working on Unix and MacOsX):

  1. Determine how many nodes you want to start the cluster with: 2, 4, 6, 8
    • Note: the number of the nodes should be >= 2
  2. Place the following files respectively to your Virtuoso installation folders:
    • bin folder: the virtuoso-restore.sh file:
      #!/bin/sh
      #
      #  $Id: installation.xml,v 1.5.2.3 2010/10/07 12:44:59 source Exp $
      #
      #  (C) 2007 OpenLink Software
      #
      #  The copyright above and this notice must be preserved in all
      #  copies of this source code.  The copyright above does not
      #  evidence any actual or intended publication of this source code.
      #
      #  This is unpublished proprietary trade secret of OpenLink Software.
      #  This source code may not be copied, disclosed, distributed, demonstrated
      #  or licensed except as authorized by OpenLink Software.
      #
      #  To learn more about this product, or any other product in our
      #  portfolio, please check out our web site at:
      #
      #      http://www.openlinksw.com
      #
      #  or contact us at:
      #
      #      general.information@openlinksw.com
      #
      #  If you have any technical questions, please contact our support
      #  staff at:
      #
      #      technical.support@openlinksw.com
      #
      
      
      #
      #  Set up the script
      #
      umask 022
      
      #
      #  Optional argument which database to start/stop
      #
      DBDIR=${1-all}
      BPREF=${2-bkp}
      
      
      #
      #  Check if we need to be silent
      #
      SILENT=${SILENT-"0"}
      if test "$SILENT" = "0"
      then
          ECHO=echo
      else
          ECHO=:
      fi
      
      
      #
      #  Script information
      #
      SCRIPT_NAME=`basename $0`
      SCRIPT_VERSION="1.0"
      SCRIPT_DATE="Wed Apr 18 2007"
      
      
      #
      #  Where to look
      #
      if test "x$VIRTUOSO_HOME" = "x"
      then
          VIRTUOSO_HOME="`dirname $0`/.."
          VIRTUOSO_HOME=`cd $VIRTUOSO_HOME >/dev/null && pwd`
      fi
      if test "x$VIRTUOSO_HOME" = "x"
      then
          echo ""
          echo "***"
          echo "*** ERROR: Unable to determine value for \$VIRTUOSO_HOME"
          echo "***"
      
          exit 1
      fi
      if test \! -f "$VIRTUOSO_HOME/bin/virtuoso-iodbc-t"
      then
          echo ""
          echo "***"
          echo "*** ERROR: \$VIRTUOSO_HOME does not appear to contain standard virtuoso binary"
          echo "***"
      
          exit 1
      fi
      export VIRTUOSO_HOME SCRIPT_NAME
      
      #
      #  Read local environment
      #
      if test -f "$VIRTUOSO_HOME/virtuoso-environment.sh"
      then
          . "$VIRTUOSO_HOME/virtuoso-environment.sh"
      fi
      
      
      #
      #  Try to start the oplmgr if available
      #
      cd "$VIRTUOSO_HOME/bin"
      if test -x oplmgr
      then
          ./oplmgr +start 2>/dev/null
      fi
      
      
      #
      #  Check which database(s) to start
      #
      cd "$VIRTUOSO_HOME"
      if test "$DBDIR" = "all"
      then
          DBDIR=""
          for f in `ls -1 */virtuoso.ini 2>/dev/null`
          do
      	d=`dirname $f`
      
              if test -d "$d"
      	then
        	    DBDIR="$DBDIR $d"
      	fi
          done
      fi
      
      
      #
      #  Perform the start/stop command
      #
      for d in $DBDIR
      do
         $ECHO "Starting Virtuoso instance in [$d]"
         $VIRTUOSO_HOME/install/command-restore.sh $d $BPREF
      done
      
      exit 0
      
      
    • folder install: the files command-restore.sh and mk_cluster.sh:

      command-restore.sh

      #!/bin/sh
      #
      #  $Id: installation.xml,v 1.5.2.3 2010/10/07 12:44:59 source Exp $
      #
      #  (C) 2007 OpenLink Software
      #
      #  The copyright above and this notice must be preserved in all
      #  copies of this source code.  The copyright above does not
      #  evidence any actual or intended publication of this source code.
      #
      #  This is unpublished proprietary trade secret of OpenLink Software.
      #  This source code may not be copied, disclosed, distributed, demonstrated
      #  or licensed except as authorized by OpenLink Software.
      #
      #  To learn more about this product, or any other product in our
      #  portfolio, please check out our web site at:
      #
      #      http://www.openlinksw.com
      #
      #  or contact us at:
      #
      #      general.information@openlinksw.com
      #
      #  If you have any technical questions, please contact our support
      #  staff at:
      #
      #      technical.support@openlinksw.com
      #
      
      
      #
      #  Set up the script
      #
      umask 022
      
      
      #
      #  Optional argument which database to startup
      #
      DBDIR=${1-database}
      BPREF=${2-bkp}
      
      #
      #  Script information
      #
      SCRIPT_NAME=`basename $0`
      SCRIPT_VERSION="1.0"
      SCRIPT_DATE="Wed Apr 18 2007"
      
      
      #
      #  Where to look
      #
      if test "x$VIRTUOSO_HOME" = "x"
      then
          VIRTUOSO_HOME="`dirname $0`/.."
          VIRTUOSO_HOME=`cd $VIRTUOSO_HOME >/dev/null && pwd`
      fi
      if test "x$VIRTUOSO_HOME" = "x"
      then
          echo ""
          echo "***"
          echo "*** ERROR: Unable to determine value for \$VIRTUOSO_HOME"
          echo "***"
      
          exit 1
      fi
      if test \! -f "$VIRTUOSO_HOME/bin/virtuoso-iodbc-t"
      then
          echo ""
          echo "***"
          echo "*** ERROR: \$VIRTUOSO_HOME does not appear to contain standard virtuoso binary"
          echo "***"
      
          exit 1
      fi
      export VIRTUOSO_HOME SCRIPT_NAME
      
      
      #
      #  Settings
      #
      SLEEP=5
      INIFILE="$VIRTUOSO_HOME/bin/inifile"
      INIFLAGS="+inifile virtuoso.ini"
      
      
      #
      #  Read local environment
      #
      if test -f "$VIRTUOSO_HOME/virtuoso-environment.sh"
      then
          . "$VIRTUOSO_HOME/virtuoso-environment.sh"
      fi
      
      
      #
      #  Check if database directory exists
      #
      if test \! -d "$VIRTUOSO_HOME/$DBDIR"
      then
          echo ""
          echo "***"
          echo "***  WARNING: \$VIRTUOSO_HOME/$DBDIR does not exist"
          echo "***"
      
          exit 1
      fi
      
      
      #
      #  Check if we are allowed to startup the database at all
      #
      
      if test \! -w "$VIRTUOSO_HOME/$DBDIR"
      then
          echo ""
          echo "***"
          echo "*** ERROR: You do not have permission to startup this virtuoso instance"
          echo "***"
      
          exit 1
      fi
      
      
      #
      #  Check if there is any lock file in this directory
      #
      cd "$VIRTUOSO_HOME/$DBDIR"
      LCK=virtuoso.lck
      for i in *.lck
      do
          if test -f "$i"
          then
      	LCK="$i"
          fi
      done
      
      
      #
      #  If there is a lock file present, consider the database running
      #
      if test -f "$LCK"
      then
          echo ""
          echo "***"
          echo "***  WARNING: Lock file already exists."
          echo "***"
          echo "***  This Virtuoso instance is probably already running."
          echo "***"
      
          exit 0
      fi
      
      DBFILE=`$INIFILE $INIFLAGS +section Database +key DatabaseFile`
      TRXFILE=`$INIFILE $INIFLAGS +section Database +key TransactionFile`
      
      if test -f "$DBFILE"
      then
          echo ""
          echo "***"
          echo "***  WARNING: DataBase file already exists."
          echo "***"
          echo "***  This Virtuoso instance cannot be restored."
          echo "***"
      
          exit 0
      fi
      
      if test -f "$TRXFILE"
      then
          echo ""
          echo "***"
          echo "***  WARNING: TransactionFile file already exists."
          echo "***"
          echo "***  This Virtuoso instance cannot be restored."
          echo "***"
      
          exit 0
      fi
      
      #
      #  Start the engine
      #
      ./virtuoso +restore-backup "$BPREF" +backup-dirs backup
      if test $? -ne 0
      then
          echo ""
          echo "***"
          echo "*** ERROR: Could not start this Virtuoso Instance."
          echo "***"
      
          LOG=`$INIFILE $INIFLAGS +section Database +key ErrorLogFile`
          tail "$LOG"
      
          echo "***"
          echo "*** Please fix the above issue(s) before trying again."
          echo "***"
      
          exit 1
      fi
      
      
      #  All done
      exit 0
      
      

      mk_cluster.sh

      #!/bin/bash
      
      export NUM_NODES=${1-8}
      export NUM_DISKS=4
      
      SPACE=256
      echo diskspace per NODE = ${SPACE}M
      
      #
      #  Goto TOP
      #
      
      if test "x$VIRTUOSO_HOME" = "x"
      then
          VIRTUOSO_HOME="`dirname $0`/.."
          VIRTUOSO_HOME=`cd $VIRTUOSO_HOME >/dev/null && pwd`
      fi
      if test "x$VIRTUOSO_HOME" = "x"
      then
          echo ""
          echo "***"
          echo "*** ERROR: Unable to determine value for \$VIRTUOSO_HOME"
          echo "***"
      
          exit 1
      fi
      
      cd $VIRTUOSO_HOME
      
      #cd /opt/virtuoso
      #mkdir -p /dbs1/lod_database
      #mkdir -p /dbs2/lod_database
      #mkdir -p /dbs3/lod_database
      #mkdir -p /dbs4/lod_database
      
      #
      #  Create various directories
      #
      i=1
      while true
      do
          PORT=`expr 12200 + $i`
          CLUSTER=`printf "cluster_%02d" $i`
          HOST=`printf "Host%d" $i`
          LOD=`printf "lod-%02d" $i`
      
          echo "Creating $CLUSTER on Host $HOST port $PORT"
      
          mkdir -p "$CLUSTER"
          mkdir -p "$CLUSTER"/logs
          mkdir -p "$CLUSTER"/backup
      
          cat > "$CLUSTER"/virtuoso.ini <<-EOF_VIRTUOSO
      	;  virtuoso.ini
      	;
      	;  Configuration file for the OpenLink Virtuoso VDBMS Server
      	;
      	;
      	;  Database setup
      	;
      	[Database]
      	DatabaseFile       = virtuoso.db
      	TransactionFile    = virtuoso.trx
      	ErrorLogFile       = virtuoso.log
      	ErrorLogLevel      = 7
      	Syslog             = 0
      	TempStorage        = TempDatabase
      	FileExtend         = 200
      	MaxCheckpointRemap = 2500000
      	Striping           = 0
      
      	[TempDatabase]
      	DatabaseFile    = virtuoso.tdb
      	TransactionFile = virtuoso.ttr
      	FileExtend      = 200
      
      	;
      	;  Server parameters
      	;
      	[Parameters]
      	ServerPort                 = $PORT
      	ServerThreads              = 100
      	CheckpointInterval         = 600
      	NumberOfBuffers            = 20000
      	MaxDirtyBuffers            = 130000
      	DefaultIsolation           = 2
      	UnremapQuota               = 0
      	AtomicDive                 = 1
      	PrefixResultNames          = 0
      	CaseMode                   = 2
      	DisableMtWrite             = 0
      	;MinAutoCheckpointSize	= 4000000
      	;CheckpointAuditTrail	= 1
      	DirsAllowed                = /
      	PLDebug                    = 0
      	TestCoverage               = cov.xml
      	;Charset=ISO-8859-1
      	ResourcesCleanupInterval   = 1
      	ThreadCleanupInterval      = 1
      	TransactionAfterImageLimit = 1500000000
      	FDsPerFile                 = 4
      	MaxMemPoolSize             = 40000000
      	IndexTreeMaps              = 64
      
      	[VDB]
      	VDBDisconnectTimeout = 1000
      	ArrayOptimization    = 2
      	NumArrayParameters   = 10
      
      	[Client]
      	SQL_QUERY_TIMEOUT  = 0
      	SQL_TXN_TIMEOUT    = 0
      	SQL_ROWSET_SIZE    = 10
      	SQL_PREFETCH_BYTES = 12000
      
      	[AutoRepair]
      	BadParentLinks = 0
      	BadDTP         = 0
      
      	;[Striping]
      	;Segment1 = ${SPACE}M, /dbs1/lod_database/${LOD}-sl1.db = q1, /dbs2/lod_database/${LOD}-sl2.db = q2, /dbs3/lod_database/${LOD}-sl3.db = q3, /dbs4/lod_database/${LOD}-sl4.db = q4
      
      	[Replication]
      	ServerName   = $LOD
      	ServerEnable = 1
      
      	[!URIQA]
      	DefaultHost = localhost:8890
      EOF_VIRTUOSO
      
      if test $i -eq 1
      then
          cat >> "$CLUSTER"/virtuoso.ini <<-EOF_VIRTUOSO_HTTP
      
      	[HTTPServer]
      	ServerPort                  = 8890
      	ServerRoot                  = $VIRTUOSO_HOME/vsp
      	ServerThreads               = 15
      	MaxKeepAlives               = 10
      	KeepAliveTimeout            = 10
      	MaxCachedProxyConnections   = 10
      	ProxyConnectionCacheTimeout = 10
      	DavRoot                     = DAV
      	HTTPLogFile                 = logs/http.log
      EOF_VIRTUOSO_HTTP
      fi
          # Create cluster control
          cat > "$CLUSTER"/cluster.ini <<-EOF_CLUSTER
      	[Cluster]
      	Threads = 50
      	ThisHost = $HOST
      	Master = Host1
      	ReqBatchSize = 10000
      	BatchesPerRPC = 4
      	BatchBufferBytes = 20000
      	LocalOnly = 2
      	MaxKeepAlivesMissed = 1000
      	MaxHosts = 100
      
      EOF_CLUSTER
          let n=0
          while true
          do
      	let n+=1
      	CLPORT="`expr 22200 + $n`"
      	echo "Host$n = $CLPORT" >> "$CLUSTER"/cluster.ini
      	if test $n -ge $NUM_NODES
      	then
      	    break
      	fi
          done
      
          # Linking virtuoso executable in place
          cd "$CLUSTER"
          ln -sf "$VIRTUOSO_HOME"/bin/virtuoso-iodbc-t virtuoso
          cd ..
      
          let i+=1
          if test $i -gt $NUM_NODES
          then
      	break
          fi
      done
      
      exit 0
      
      
  3. If you are using EC2 instance, make sure you set the environment by running:
    . ./virtuoso-environment.sh
    
  4. Run the mk_cluster.sh file from the .\install folder with option number of the desired nodes:
    ..# bash
    ..# . ./mk_cluster.sh  4
    ....
    Creating cluster_01 on Host Host1 port 12201
    Creating cluster_02 on Host Host2 port 12202
    Creating cluster_03 on Host Host3 port 12203
    Creating cluster_04 on Host Host4 port 12204
    
    

2.14.1.1. Backup and Restore

Backup

There are 2 ways to backup Virtuoso Cluster DB:

  1. Backup every node using back_online

    For example, for every cluster in its backup folder execute:

    backup_online ('dbp', 10000000, 0, 'backup');
    
  2. Make backup of all nodes at once using cl_exec. For example, execute:
    cl_exec ('backup_online ('dbp', 10000000, 0, 'backup')', vector ('dbp', 0, 10000000, './backup'));
    

Restore from Backup

After backup is done for every node in its backup folder, in order to restore, execute from the ..\bin folder the virtuoso-restore.sh:

# . ./virtuoso-restore.sh all dbp

where the second parameter is the restore prefix.



2.14.2. Cluster Config

  1. Set "FAST_START=1" in the viruoso-start.sh file and then run:
    viruoso-start.sh
    

    or

  2. Execute the following line:
    # /etc/init.d/virtuoso start
    Starting OpenLink Virtuoso:                                [  OK  ]
    
  3. In order to check the nodes, connect to port 12201 using the ISQL tool:
    isql 12201
    
  4. To check the cluster status, execute the following command:
    SQL> status('cluster');
    REPORT
    VARCHAR
    _______________________________________________________________________________
    
    Cluster 4 nodes, 293 s. 0 m/s 0 KB/s  0% cpu 0%  read 0% clw threads 1r 0w 0i buffers 1781 0 d 0 w 0 pfs
    
    1 Rows. -- 4 msec.
    

See Also:

Setting up and operating Virtuoso on a cluster.

Virtuoso Cluster Programming

Virtuoso Cluster Fault Tolerance.