Giter Site home page Giter Site logo

template-library-grid's People

Contributors

guillaume-philippon avatar jouvin avatar jrha avatar pansanel avatar pigay avatar rhumroux avatar sartiran avatar stdweird avatar victor-mendoza avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

template-library-grid's Issues

Global variables set to null by default

In many places, we see global variables initialized by default (?=) to 'null' and we would like to understand why is it so. An example:

variable WN_CONFIG_SITE ?= null;

at the beginning of personality/wn/cloning/standard. Due to the major change in the way null is considered with PAN 10.7, it prevents site level default assignment from working. For example, if you subsequently set:

variable WN_CONFIG_SITE ?= 'config/wn';

at the level of the site, it won't work.

[ERROR] /usr/sbin/glite-info-static-create does not exist

Hello,

when I run a fresh install of a CREAM-CE, I get the following error on the gip2 component.
I am using quattor 14.2.1 for deployment.

If I run it a second time, then the error disappears.
Could you please help me with this issue?
Following is the log I see:

[INFO] running component: gip2

[INFO] 1File /var/lib/bdii/gip/plugin/glite-info-dynamic-ce removed (not part ncm-gip2 configuration).
Insecure dependency in unlink while running with -t switch at /usr/lib/perl/NCM/Component/gip2.pm line 174.
Insecure dependency in unlink while running with -t switch at /usr/lib/perl/NCM/Component/gip2.pm line 174.
[INFO] 1File /var/lib/bdii/gip/plugin/glite-info-dynamic-scheduler-wrapper removed (not part ncm-gip2 configuration).
Insecure dependency in unlink while running with -t switch at /usr/lib/perl/NCM/Component/gip2.pm line 174.
Insecure dependency in unlink while running with -t switch at /usr/lib/perl/NCM/Component/gip2.pm line 174.
[INFO] Setting owner, group and permissions on directory /etc/bdii/gip and its contents
[INFO] Setting owner, group and permissions on directory /var/lib/bdii/gip/plugin and its contents (recursive)
[INFO] Setting owner, group and permissions on directory /var/lib/bdii/gip/provider and its contents (recursive)
[INFO] Setting owner, group and permissions on directory /var/lib/bdii/gip/ldif and its contents (recursive)
Insecure dependency in chown while running with -t switch at /usr/lib/perl/NCM/Component/gip2.pm line 463.
Insecure dependency in chown while running with -t switch at /usr/lib/perl/NCM/Component/gip2.pm line 463.
Insecure dependency in chown while running with -t switch at /usr/lib/perl/NCM/Component/gip2.pm line 463.
[INFO] Setting owner, group and permissions on directory /var/lib/bdii/gip/tmp/gip and its contents (recursive)
[INFO] Setting owner, group and permissions on directory /var/lib/bdii/gip/cache/gip and its contents (recursive)
[ERROR] /usr/sbin/glite-info-static-create does not exist
[INFO] configure on component gip2 executed, 1 errors, 0 warnings

vo/functions.pan: is the removal of CVMFS mount points really correct?

Commit b4f3000 introduced a change to directory created/configured as part of the VO configuration on UI and WN, removing the CVMFS mount points from the list. The commit message speaks only about the UI being affected by the change but it seems to affect also WNs and I am surprised that CVMFS can work without the mount point being created before (at least it has not always been the case). @Pansanel do you confirm that this change is find on newly installed WN?

GIP_CE_MAUI_PLUGIN_DEFAULTS_FILE owner/group

The owner and group of this file are set to root and 600 permissions in the template ce-maui-plugin-defaults.pan. I get a 'permission denied error' for the user ldap.
Why not to set the permissions to 644 by default?

(quattor-14.10) umd3 dpm problem for dpm-disk without xrootd

Hi,

We encountered a problem with our dpm-disk servers when upgrading to recent packages (dpm-1.8.9).

For some reason, DPM certs directory /etc/grid-security/dpmmgr/ is not created. We suspect that this is because we don't have xrootd protocol enabled on our SE.

This directory was present on our SE head node, maybe because of a previous configuration.

It resulted in "no space left on device" errors when putting a file on the SE.

Manually copying host cert in /etc/grid-security/dpmmgr/dpmcert.pem and /etc/grid-security/dpmmgr/dpmkey.pem solved the problem:

mkdir -p /etc/grid-security/dpmmgr/
cp /etc/grid-security/hostcert.pem /etc/grid-security/dpmmgr/dpmcert.pem
cp /etc/grid-security/hostkey.pem /etc/grid-security/dpmmgr/dpmkey.pem
chown -R dpmmgr:dpmmgr /etc/grid-security/dpmmgr/

Regards,

Pierre

Configure HEP_OSlibs in a common template

As discussed in #147, both personality/ui/rpms/config and personality/wn/rpms/config do essentially the same thing when it comes to installing the appropriate HEP_OSlibs package, this duplicated code would be better off in a single common template.

Both features/hep_oslibs/config or features/wlcg/config were put forward as options.

VOBOX missing

In August, we received a report from Maarten Litmaath (grid expert) that Quattor sites were still running unsupported VOBOX, in particular with an obsolte VOMS client (no longer working with SHA2 certificates). In fact we forgot to release the umd-3 support for this machine type. Ideally, should be part of the upcoming 14.8 release (will not break anything anyway).

Maarten's recommandations to build the VOBOX are:

the WLCG VOBOX is documented here:

https://twiki.cern.ch/twiki/bin/view/LCG/WLCGvoboxDeployment

The wlcg-vobox meta package depends on these requirements:

edg-mkgridmap
emi-ui
glite-yaim-clients (already covered by emi-ui)
gsi-openssh-clients
gsi-openssh-server
lcg-vobox

The last package can be taken from the WLCG repo:

http://linuxsoft.cern.ch/wlcg/

WN software/components/symlink/links is undefined

I get this error while compiling a basic grid/wn machine type.

[panc] element at /software/components/symlink/links is undefined

I understand this might be assigned to some variable that is empty at the moment, therefore the list of links is empty.

Could you help me solving this?
Cheers

add configuration options

could the configuration of the groups be extended with the setting of priorities for the htcondor groups? GROUP_PRIO_FACTOR_group_*
Also NEGOTIATOR_ALLOW_QUOTA_OVERSUBSCRIPTION.

published info with htcondor

The bdii plugins are correct, and shows the right number of GlueCEStateRunningJobs when using htcondor batch system (/var/lib/bdii/gip/plugin/lcg-info-dynamic-scheduler-wrapper via /usr/libexec/lcg-info-dynamic-scheduler), but it is not being updated in the bdii slapd.
Bdii log shows no error. Number of GlueCEStateRunningJobs is always kept to 0.

WMS configuration fixes

Upgrading a WMS to 14.6.0 umd-3 templates, a few minor problems where found.

Would be good to merge before 14.8.0 release.

three VOMS servers for each VO supported by UK VOMS servers

There are currently three VOMS servers (voms.gridpp.ac.uk, voms02.gridpp.rl.ac.uk, voms03.gridpp.rl.ac.uk) used by various VOs supported by the UK VOMS service.

Only one VO (t2k.org) is correctly and fully configured within the template-library-grid/vo/params/t2k.org.pan template, the rest of them (babar, camont, comet.j-parc.jp, gridpp, mice, minos.vo.gridpp.ac.uk, na62.vo.gridpp.ac.uk, neurogrid.incf.org, ngs.ac.uk, pheno, snoplus.snolab.ca, supernemo.vo.eu-egee.org, vo.helio-vo.eu, vo.landslides.mossaic.org, vo.londongrid.ac.uk, vo.northgrid.ac.uk, vo.southgrid.ac.uk) are using only voms.gridpp.rl.ac.uk VOMS server.

This need to be corrected in order to improve the service resilience (there are situations when voms.gridpp.ac.uk server doesn't serve requests properly)

Thanks,
Catalin

Support for new CE types (ARC CE, HTCONDOR CE)

MAUI is not supported anymore and is not available for CentOS 7. Grid sites should migrate either to SLURM or HTCONDOR batch system. In the same time, site admins should consider to use ARC CE (or HTCONDOR CE) if they plan to deploy HTCONDOR (most sites using HTCONDOR have deployed ARC CE).
Would it be possible to have a set of template for ARC CE?

features/nfs: wrong handling of nfs version

Hi,

in e.g. features/nfs/client/autofs.pan (lines 51-59), the handling of nfsversion does not work for SL 6. In SL 6, the default nfs version is 4, but the template assumes that the default is 3 (as it was in SL 4 and 5).

Best regards,
Andreas

Grid templates not useable in Aquilon

Unfortunately we are unable to use the grid templates as intended as we are unable to include "defaults/grid/config". In particular we are currently trying to use the templates to create a BDII

This is due to the use of DB_MACHINE which is not used in Aquilon and I can't see a straight forward way of constructing this.

UI machine type

Hi,

I have tried to fresh install a UI machine, and I get this dependency errors:

[INFO] running component: spma

[WARN] synchronisation with upstream produced warnings: Error: Package: java-1.6.0-sun-compat-1.6.0.24-3.sl5.jpp.x86_64 (@x86_64_sl6_glite)
Requires: jdk = 2000:1.6.0_24-fcs
Removing: 2000:jdk-1.6.0_24-fcs.x86_64 (@emi_3_0_updates)
jdk = 2000:1.6.0_24-fcs
Updated By: 2000:jdk-1.7.0_45-fcs.x86_64 (x86_64_sl6)
jdk = 2000:1.7.0_45-fcs
Available: 2000:jdk-1.5.0_05-fcs.i586 (x86_64_sl6_glite)
jdk = 2000:1.5.0_05-fcs
Available: 2000:jdk-1.5.0_14-fcs.i586 (x86_64_sl6_glite)
jdk = 2000:1.5.0_14-fcs
Available: 2000:jdk-1.6.0-fcs.i586 (x86_64_sl6_glite)
jdk = 2000:1.6.0-fcs
Available: 2000:jdk-1.6.0_03-fcs.x86_64 (x86_64_sl6_glite)
jdk = 2000:1.6.0_03-fcs
Available: 2000:jdk-1.6.0_04-fcs.i586 (x86_64_sl6_glite)
jdk = 2000:1.6.0_04-fcs
Available: 2000:jdk-1.6.0_07-fcs.i586 (noarch)
jdk = 2000:1.6.0_07-fcs
Available: 2000:jdk-1.6.0_26-fcs.x86_64 (x86_64_sl6)
jdk = 2000:1.6.0_26-fcs
Error: Package: canl-java-1.3.0-1.el6.noarch (@emi_3_0_updates)
Requires: bcprov = 1.46
Removing: bouncycastle-1.46-1.el6.noarch (@x86_64_sl6_glite)
bcprov = 1.46-1.el6
Updated By: bouncycastle-1.46-4.jpp6.noarch (x86_64_sl6_glite)
Not found
Available: bouncycastle-1.45-6.el5.x86_64 (emi_1_0_third-party)
bcprov = 1.45-6.el5
Available: bouncycastle-1.43-1.jpp5.noarch (x86_64_sl6_glite)
Not found
Available: bouncycastle-1.45-1.jdk1.6.tomcat.noarch (x86_64_sl6_glite)
Not found
Error: Package: sun-mail-1.4-3jpp.noarch (x86_64_sl6_glite)
Requires: gnu-crypto-sasl-jdk1.4
Available: gnu-crypto-sasl-jdk1.4-2.1.0-2jpp.noarch (emi_3_0_updates)
gnu-crypto-sasl-jdk1.4 = 2.1.0-2jpp

[WARN] Command output: Loaded plugins: changelog, kernel-module, priorities, protectbase, tsflags,
: versionlock
...

I am installing SL630 os, but the dependencies refer to sl5. I would appreciate if you could help me solving this issue.
Thank you very much.

GIP CE: finalize GLUE2 support

After introducing GLUE2 support for CE information in 14.10 (#45), finalize this support. This is mainly:

Additionally:

  • Fix GLUE1 support for cluster mode (too many information published on LRMS-only node)
  • Configure /etc/scheduler.conf and/or /etc/bdii/gip/lcg-info-dynamic-scheduler-pbs.conf as done by YAIM in config_gip_sched_plugin_pbs (part of package glite-yaim-torque-utils), using CAF::TextRender
  • Remove RPMs from this template

Requires #114.

VOMS server: manage voms-admin configuration files

Currently, the voms-admin configuration files for each VO (located in /etc/voms-admin) are unmanaged and thus reflecting default values. It would be good to allow some control on service.properties at least as this file contains everything related to the membership renewal policy.

APEL box: configuration wrong or incomplete

Hi!

It seems that the configuration of an UMD-3 APEL box is incomplete or wrong. The mysql configuration of a machine built with "machine-types/grid/apel" tries to use
/usr/share/glite-apel-core/scripts/apel-schema.sql which is not available.

I get this output:

[INFO] running component: mysql

[INFO] Checking if MySQL service (mysqld) is enabled and started...
[INFO] Configuring database accounting
[WARN] Error executing initialization script
(/usr/share/glite-apel-core/scripts/apel-schema.sql) for database accounting
[ERROR] Error creating database accounting on server xxx
[INFO] configure on component mysql executed, 1 errors, 1 warnings

It seems that templates in personality/apel and/or features/accounting/apel have been
copied from the EMI-2 branch without applying all necessary configuration changes.

Best regards,
Andreas

ARGUS: Missing information in BDII

Hi,

since the upgrade of our Argus server, I get an error from our NGI's nagios tests (see below). Comparing the templates from UMD-3 branch with those from EMI-2 branch, I see that some lines in "personality/argus/service" have been dropped:

# Configure BDII or GRIS, whatever is appropriate
include { 'glite/bdii/service' };

# Configure the information provider.
include { 'common/gip/base' };
include { 'common/gip/argus' };

Therefore nothing about argus is published in BDII.

Best regards,
Andreas

---8<-----------------------------------------
Current Status:
CRITICAL
(for 3d 0h 21m 44s)
Status Information: BDII_ENTRIES_NUM CRITICAL - 0 entries found
Performance Data: time=0.360269069671631s;;
Current Attempt: 1/1 (HARD state)
Last Check Time: 06-16-2014 18:21:21
Check Type: ACTIVE
Check Latency / Duration: 2.415 / 0.466 seconds
Next Scheduled Check: 06-17-2014 18:21:21
Last State Change: 06-13-2014 18:21:18
Last Notification: 06-15-2014 18:21:29 (notification 3)
Is This Service Flapping? N/A
In Scheduled Downtime?
NO
Last Update: 06-16-2014 18:42:58 ( 0d 0h 0m 4s ago)
---8<-----------------------------------------
(https://midmon.egi.eu/nagios/cgi-bin/extinfo.cgi?type=2&host=grid-mon.physik.rwth-aachen.de&service=eu.egi.sec.Argus-EMI-1)

component-gsissh error (quattor 14.2.1) for vobox umd-3 : /usr/setup/gsi_openssh_setup/setup-openssh isn't executable

On a sl6 vobox, cluster emi2 or umd-3, templates from april 2014 and rpm, the gsissh component has an error : /usr/setup/gsi_openssh_setup/setup-openssh isn't executable

[root@lyogrid05 ncm]# rpm -qa|grep gsi|grep ssh
ncm-gsissh-14.2.1-1.noarch
gsi-openssh-5.3p1-10.el6.x86_64
gsi-openssh-server-5.3p1-10.el6.x86_64
gsi-openssh-clients-5.3p1-10.el6.x86_64

root@lyogrid05 ncm]# ncm-ncd --configure gsissh

[INFO] NCM-NCD version 14.2.1 started by root at: Mon Sep 1 16:22:36 2014
[INFO] executing configure on components....

[INFO] running component: spma

[INFO] configure on component spma executed, 0 errors, 0 warnings

[INFO] running component: gsissh

2014/08/28-11:50:18 [INFO] updated /usr/etc/ssh//ssh_config
2014/08/28-11:50:18 [ERROR] /usr/setup/gsi_openssh_setup/setup-openssh isn't executable
2014/08/28-11:50:18 [INFO] configure on component gsissh executed, 1 errors, 0 warnings
[INFO] configure on component gsissh executed, 1 errors, 0 warnings

[INFO] Errors while configuring gsissh (1)
[ERROR] 1 errors, 0 warnings executing configure

htcondor repo

template template-library-grid/repository/config/htcondor.pan
Currently the repository defined by HTCONDOR_YUM_REPOSITORY_NAME is not being used. According to line 64:

variable YUM_HTCONDOR_REPOSITORY_LIST ?= {
if ( !is_null(YUM_HTCONDOR_REPOSITORY_LIST) ) {
append('htcondor');
};
};

only 'htcondor' is appended.
Shouldn't it be append(HTCONDOR_YUM_REPOSITORY_NAME); instead?

Cheers

ncm-vomsclient produces wrong lsc files

Hello,

I have a problem with the lsc files produced by ncm-vomsclient (14.2.1). When glexec uses an argus server I get this error message in the pepd log for the new SHA-2 certificates:

2014-09-02 22:55:23.860Z - ERROR [AbstractX509PIP] - LSC signature validation failed: matching AA cert CN=voms2.cern.ch,OU=computers,DC=cern,DC=ch fails signature verification.
2014-09-02 22:55:23.860Z - ERROR [PEPDaemonRequestHandler] - Error processing PIP: LSC signature validation failed: matching AA cert CN=voms2.cern.ch,OU=computers,DC=cern,DC=ch fails signature verification.

The reason for this error seems to be a missing new-line character at the end of the lsc file. Adding a new-line at the end of the file manually solves the problem.

I would say this is a critical bug since all sites should be able to deal with SHA-2 certificates by mid September.

Best regards,
Andreas

ncm-xrootd 14.5.0: local redirector should be started first

In ncm-xrootd14.5.0, xrootd instances are (re)started in the instance name alphabetical order (hash key alphabetical order). This is not working anymore in recent versions of xrootd because the local redirector must be started first: federation redirectors startup is not completed until it has successfully established a connection with the local redirector.

GIP CE: fix remaing FIXME lines

The new GLUE2 compliant GIP CE configuration template (see #110) has a few FIXME lines left for minor issues. Would be good to process them at some point...

template personality/cream_ce/templates/cream-config.templ

JOB_SUBMISSION_MANAGER_SCRIPT_PATH should take /etc/glite-ce-cream-ce-utils/glite_cream_load_monitor.conf instead of /etc/glite-ce-cream-utils/glite_cream_load_monitor.conf, because of the configuration in personality/cream_ce/monitor.pan which is using hard-coded path to /etc/glite-ce-cream-ce-utils/glite_cream_load_monitor.conf

Env variable SASL_PATH defined to an invalid value

features/globus/env.pan improperly defines $SASL_PATH to /usr/lib/sasl (which doesn't exist, SASL is installed in /usr/lib64/sasl2). This is probably a leftover from very very ancient time, when Globus was providing a non standard SASL version.
This doesn't seem to have any impact on the grid MW itself (else we'll have had complaints!). But it breaks any attempt to use SASL on the machine, for example with Kerberos (I found troubleshooting problems to use GSSAPI to authenticate against a LDAP server)?

14.6.0: WMS without a LB not working

Templates for WMS in 14.6.0 works only with a WMS/LB configuration. A WMS only machine fails to register jobs because it fails to communicate with the LB. Looks like there are some missing services.

condor_submit fails with SMPgranularity parameter

I have configured a CREAM-CE with htcondor batch system.

When I include the variable:
SMPgranularity = $CORES;
in my JDL code, I get from CREAM an error of the condor_submit script
05 May 2017 16:25:35,580 INFO AbstractJobExecutor - JOB CREAM389568339 STATUS CHANGED: PENDING => ABORTED [description=submission to BLAH failed [retry count=3]] [failureReason=BLAH error: submission command failed (exit code = 1) (stdout:Usage: /usr/libexec/condor_submit.sh -c [-i ] [-o ] [-e ] [-v ] [-s <yes | no>] [-- command_arguments]-) (stderr:/usr/libexec/condor_submit.sh: illegal option -- S-) N/A (jobId = CREAM389568339)] [localUser=atl124]

the same JDL works when removing that parameter.
The CONDOR_CONFIG ['multicore'] is set to true.

The JDL:
[
Executable="$script_file";
InputSandbox = {"$script_file"};
StdOutput = "stdout.out";
StdError = "stderr.out";
OutputSandbox = {"stdout.out", "stderr.out"};
#SMPgranularity = $CORES;
CpuNumber = $CORES;
OutputSandboxBaseDestURI= "gsiftp://localhost";
]

Problem with publication information by BDII-Local

I got a alert on

midmon.egi.eu

on my bdii about information the bdii publish

CRITICAL - errors 78, warnings 26, info 52
Summary per type of error, warning and info message:
E022 - Default value published (GLUE2ComputingShareWaitingJobs): 26
E023 - Default value published (GLUE2ComputingShareEstimatedAverageWaitingTime): 26
E024 - Default value published (GLUE2ComputingShareEstimatedWorstWaitingTime): 26
I046 - Number of seconds higher than 1 million (GLUE2ComputingShareEstimatedAverageWaitingTime): 26
I047 - Number of seconds higher than 1 million (GLUE2ComputingShareEstimatedWorstWaitingTime): 26
W025 - Incoherent number of total jobs (GLUE2ComputingShareTotalJobs): 26

cream ce is not working with present templates

ncm-ncd is failing for cream-ce.

The problem is due to inconsistency:
in personality/cream_ce/config.pan:

'/software/components/glitestartup/active' = false;

and in

features/lb/locallogger.pan:

'/software/components/filecopy/dependencies/post' = push('glitestartup');

commenting the line in features/lb/locallogger.pan solves the problem for cream ce but was not tested for wms and lb.

Why is VOMS_SERVER_DN a list?

It is written like a dict (or nlist given it's age), is this a typo or intentional?
It certainly makes it quite annoying to use.

Missing symbolic link

They are a missing link or a misconfigure file inside the bdii configuration.

The configuration file

/etc/bdii/gip/lcg-info-dynamic-scheduler-pbs.conf

claim

[Main]
static_ldif_file: /var/glite/static-file-all-CE-pbs.ldif
static_glue2_ldif_file_computingshare: /var/glite/ComputingShare.ldif

but the file

/var/glite/ComputingShare.ldif

are in fact in

/var/lib/bdii/gip/ldif/ComputingShare.ldif

So either the configuration file need to change or quattor need to put a symbolic link between those two file

Without this the publication failed....silently.

Tidy up ARGUS templates

As noted in #134, the ARGUS templates are quite messy, with huge blocks of filecopy and usage of replace() where format() would be better used. They work fine, but let's try to make them nicer.

collisions group id

On a fresh install with centos7 (7.6.1810) which uses the uses infosys and edginfo users, I find collisions in the groupid.

[INFO] running component: accounts

[INFO] System has 33 accounts in 57 groups
[WARN] No groups found for root in the system. Assigning default ones: root, adm, bin, daemon, sys, disk
[INFO] Changing gid of group nagios to 76389
[ERROR] Collision found between groups ssh_keys and infosys for id 997
[ERROR] Collision found between groups edginfo and input for id 999
[ERROR] System would be inconsistent. Leaving without changing anything
[INFO] configure on component accounts executed, 3 errors, 1 warnings
[INFO] EVENT: accounts modified file /etc/login.defs

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.