Giter Site home page Giter Site logo

template-library-core's Introduction

Templates for configuring a specific Quattor release

This repository provides all the templates required to configure a specific version of the Quattor client and the related configuration modules used by other parts of the template library. There is one tag per Quattor version.

This repository is intended to be used in conjunction with other parts of the template library (standard, os, grid...) to actually configure a system. To help putting together everything needed in a way directly usable in a real configuration, it is recommended to follow this procedure.

Note: despite what suggest the script name/location, it is recommended to use this script whether you are using SCDB or not to get all parts of the template library organized in a proper way.

template-library-core's People

Contributors

alvarosimon avatar apdibbo avatar drossy avatar dzila avatar jouvin avatar jrha avatar kwaegema avatar ned21 avatar piojo avatar piojo-zz avatar rhumroux avatar stdweird avatar ulrich1919 avatar victor-mendoza avatar wdpypere avatar wpoely86 avatar xaf avatar

Stargazers

 avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

template-library-core's Issues

add network device types

  • is_interface_device from metaconfig ctdb service
  • usable_network_interface from metaconfig ptpd service

Add /system/vm in the tree

In order to manage the VM part in the profile, I would like to add a new type 'vm' under /system/ (/system/vm) so we can hook every virtualization technology underneath (/system/vm/opennebula & /system/vm/openstack).
This is meant to avoid having every existing virtualization technology under /system itself

What is oldnames intended for?

quattor/types/system defines a list of structure_oldname at /system/oldnames, there is no comment or annotation to indicate the purpose of this field, does anyone know what it is/was for?
If it isn't used, could it be be removed?

add valid_interface type

Following is already used a lot in templates

type valid_interface = string with exists ("/system/network/interfaces/" + SELF);

Expected behaviour of push/npush/npush_if

While working on #102, i stumbled on some ill defined behaviour of push (which also exists in npush and push_if)

object template push;

include 'pan/functions';

"/functions/push_dml_empty/data" = {
    foreach(idx;id;list(1,2,3)) {
        push(id);
    };
};

"/functions/push_dml_nonempty/data/0" = 0;
"/functions/push_dml_nonempty/data" = {
    foreach(idx;id;list(1,2,3)) {
        push(id);
    };
};

current master returns

{
  "functions": {
    "push_empty": {
      "data": [
        3
      ]
    },
    "push_nonempty": {
      "data": [
        0,
        1,
        2,
        3
      ]
    }
  }
}

so there is a significance difference between push depending on the the existence of SELF (the reimplementation using merge as in the PR suffers from similar yet different problems. (chagning the code and ending with SELF; gives different issues altogether, but they are still different).

my question is: how to handle this. keep the old code, but warn in the doc that push/npush/push_if have undesired sideeffects in DML blocks?

some sanity could be restored by using append in push/push_if, but there is no equivalent of append for dicts (in particular, merge with single argument to update SELF is not supported; see quattor/pan#103).

Template validation as part of pull requests

As discussed during the workshop, it'd be nice to have in all template-library-xxx repositories a validation test associated with pull requests. The initial implementation of the test could be the execution of create-vanilla-SCDB.sh (in SCDB repo). This script pulls down everything needed to compile the examples and compile them. We may just have to check that the script properly report a failure status.

14.6.0-rc1 issues

This issue lists the main issues found in 14.6.0-rc1 (reference this issue in blocking issues).

volgroup_required option

In Quattor 14.2.1 The Quattor file / aii / ks / config.pan ends with the line:

"/system/aii/osinstall/ks/volgroup_required" ?= true;

But from with Quattor 14.6, this line has turned into:

"/system/aii/osinstall/ks/volgroup_required" = true;

This option is no longer overloaded by SL6 variant, so that the volgroup always appears in the ks file and therefore causes an error.

provide `structure_monitoring`

the current stucture_system has an optional monitoring attribute with type structure_monitoring. This type however is not provided as part of this repo, and most likely comes from from the include 'quattor/monitoring/lemon';, which is also not part of this repository.

volgroup_required option

Since the 14.8.0 version of the file quattor/aii/ks/config ends with the line:

"/system/aii/osinstall/ks/volgroup_required" = true;

Ks files generated and always have the volgroup inserted rows , also for sl6 and 7 which causes an error in the installation.

If I change this line as follows:

"/system/aii/osinstall/ks/volgroup_required" ?= true;

Then the files to sl6 ks and 7 no longer have these insertions, which is okay.

Regards.

'-selinux*' in ks/config.pan

Problem in ks/config.pan :
When selinux is disable, '-selinux*' is append in yum install section of the ks file and generate this /tmp/post-log.log file :

Begin of post section
+ cat
+ mkdir -p /tmp/aii/yum/repos
+ cat
+ cat
+ rpm -e --nodeps kernel-firmware
error: package kernel-firmware is not installed
+ rpm -e --nodeps yum-conf
error: package yum-conf is not installed
+ yum -c /tmp/aii/yum/yum.conf -y install curl lsof openssh openssh-server perl-AppConfig perl-CDB_File perl-Crypt-SSLeay perl-DBI perl-GSSAPI perl-IO-String perl-libwww-perl perl-Pod-POM perl-Template-Toolkit perl-URI perl-XML-Parser yum-plugin-priorities yum-plugin-versionlock wget '-selinux*' NetworkManager bind-utils kernel-devel ncm-grub-15.12.0-1.noarch ncm-spma-15.12.0-1.noarch 'kernel*-3.10.0-327.10.1.el7.x86_64' kernel-headers-3.10.0-327.10.1.el7.x86_64 ncm-cdispd cdp-listend ncm-query ncm-ncd
Command line error: no such option: -s
Usage: yum [options] COMMAND

List of Commands:

check          Check for problems in the rpmdb
check-update   Check for available package updates
clean          Remove cached data
deplist        List a package's dependencies
distribution-synchronization Synchronize installed packages to the latest available versions
downgrade      downgrade a package
erase          Remove a package or packages from your system
fs             Creates filesystem snapshots, or lists/deletes current snapshots.
fssnapshot     Creates filesystem snapshots, or lists/deletes current snapshots.
groups         Display, or use, the groups information
help           Display a helpful usage message
history        Display, or use, the transaction history
info           Display details about a package or group of packages
install        Install a package or packages on your system
list           List a package or groups of packages
load-transaction load a saved transaction from filename
makecache      Generate the metadata cache
provides       Find what package provides the given value
reinstall      reinstall a package
repo-pkgs      Treat a repo. as a group of packages, so we can install/remove all of them
repolist       Display the configured software repositories
search         Search package details for the given string
shell          Run an interactive yum shell
swap           Simple way to swap packages, instead of using shell
update         Update a package or packages on your system
update-minimal Works like upgrade, but goes to the 'newest' package match which fixes a problem that affects your system
updateinfo     Acts on repository update information
upgrade        Update packages taking obsoletes into account
version        Display a version for the machine and/or available repos.


+ fail 'Unable to install packages'
/tmp/ks-script-2wwO8v: line 253: fail: command not found
+ chmod +x /etc/rc.d/init.d/ks-post-reboot
+ ln -s /etc/rc.d/init.d/ks-post-reboot /etc/rc.d/rc3.d/S86ks-post-reboot
+ wget -q --output-document=- http://lapp-quattor.in2p3.fr/cgi-bin/aii-installack.cgi
/tmp/ks-script-2wwO8v: line 257: wget: command not found
+ echo 'End of post section'
End of post section
+ sleep 0

quattor-client: remove extra dependencies

As noted in quattor/release#90 we include extra packages on quattor clients.
In this repository this is handled by quattor/client/rpms.pan.

These are:

  • yum-priorities (xL5)
  • yum-plugin-priorities (! xL5)
  • yum-plugin-versionlock
  • perl-AppConfig
  • python-elementtree (xL5)

We should add these as dependencies of the things that need them and remove them from this template.

14.5.0-rc2 issues

Hi,

Here is a list of issues I discovered so far. Some of them may be RAL specific

  1. duplicate function definition: copy_network_params

this is caused by functions/network.tpl conflicting with site/functions. Solution: comment site/function

[panc] evaluation error [/home/tier1/dzila/wc1/cfg/quattor/14.5.0-rc2/quattor/schema.pan:42.1-42.38]
include { 'monitoring/lemon/schema' };
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[panc] cannot locate template named 'monitoring/lemon/schema'

Solution: comment the include { 'monitoring/lemon/schema' };

panc] referenced type 'structure_monitoring' is not defined
[panc] >>> call stack trace
[panc] >>> [/home/tier1/dzila/wc1/cfg/quattor/14.5.0-rc2/quattor/schema.pan:374.1-395.2]

Solution: comment then relevant line from quattor/schema.pan

cannot locate template named 'hardware/functions'

Solution: svn cp from 14.4.0-rc2 (which in turh I think I had copied from standard)

[panc] evaluation error [/home/tier1/dzila/wc1/cfg/sites/ral-tier1/site/config.tpl:64.1-64.25]
include { 'os/version' };
^~~~~~~~~~~~~~~~~~~~~~~~~
[panc] cannot locate template named 'os/version'

Solution: copy from 14.4.0-rc2

  1. [panc] evaluation error [/home/tier1/dzila/wc1/cfg/quattor/14.5.0-rc2/quattor/client/rpms.pan:15.1-15.44]
    include { 'rpms/package_default_versions' };
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    [panc] cannot locate template named 'rpms/package_default_versions'

Compilation proceeds if the relevant line is commented, but then you end up with a system that does not update quattior client version!

quattor/aii/pxelinux/config.pan : Compilation fails, had to copy the one from the standard

/home/tier1/dzila/wc1/cfg/quattor/14.5.0-rc2/components/spma/config-rpm.pan

record definition contains unexpected field(s): packages
[panc] element path: '/software/components/spma'
[panc] type: 'component_spma_type' [/home/tier1/dzila/wc1/cfg/quattor/14.5.0-rc2/components/spma/schema.pan:70.28-98.1]
[panc] path '/software/components/spma' bound to type component_spma_type in [/home/tier1/dzila/wc1/cfg/quattor/14.5.0-rc2/components/spma/schema.pan:100.36-100.54]

Wrong prefix? It looked like
unique template components/spma/config-rpm;

Set prefix to root of component configuration.

prefix '/software/components/spma';
"/software/groups" ?= nlist();

Package to install

'packages' = pkg_repl("ncm-spma", "14.4.0-rc3_SNAPSHOT20140507141729", "noarch");

'packager' = 'yum';
'register_change' ?= list("/software/packages",
"/software/repositories");

I changed to

Set prefix to root of component configuration.

prefix '/software/components/spma';
'packager' = 'yum';
'register_change' ?= list("/software/packages",
"/software/repositories");

"/software/groups" ?= nlist();

Package to install

prefix '/software';
'packages' = pkg_repl("ncm-spma", "14.4.0-rc3_SNAPSHOT20140507141729", "noarch");

14.5.0-rc3 issues

I had a first test to 14.5.0-rc3 and there is still the main issue found in 14.5.0-rc2: component definitions (under components/) are not update in template-library-core. For me this comes from the change you made to push the tags, adding --tags option. According to the man page, this has the effect of pushing the tags in addition to EXPLICITLY specified refspecs. I think you should add --all to to ensure that all refspecs are pushed (or explicitly the current one if you prefer). This is the same problem that led to add a git push command in the function generating version.pan that can probably be removed...
We'll probably need a rc4...

18.6.0: linux_capability type regexp incorrect

The new linux_capability type introduced in 17.12 (d8844c1) has a a validation regexp that uses [:upper:] which is Perl specific if I'm right... At least it doesn't work properly in this context and is better replaced by [A-Z] .

add_repositories function explicitly ignores templates that don't exist

It is possible to append non-existent repositories without the compilation failing.

Example:

variable OS_REPOSITORY_LIST = {
append('quattor');
append('quattor_sl');
append('rpmforge-el6-x86_64');
append('rpmforge-extras');
append('epel-sl6-x86_64');
append('sl640_x86_64');
append('sl640_x86_64_updates');
append('sl640_x86_64_errata');
append('ral-tier1-sl5-local');
};

'/software/repositories' = add_repositories(OS_REPOSITORY_LIST);

If say epel-sl6-x86_64 is replaced with just epel which does not exist, compilation will not fail but yum will fail to install packages coming from epel.

Revert os and model fields structure archetype to Aquilon standard

In #142 we tried to remove some of the MS specific terms used for OS versions and implement a more logical structure. This was done on the supposition that:

  1. There was no dependency on those fields from shared parts of the project. This was proven untrue by the discover of the dependency on /system/archetype/name in ncm-cron.
  2. The structure was populated only by templates and thus individual sites were free to use whatever format they wish. This turned out to be incorrect since the previous October the broker had been changed to populate those fields.

At the time I thought we might be able to adopt the new structure but that subsequently also proven impossible, not only because the broker is now populating those fields which makes it a significant undertaking for us to change, but also because that version field is so important it's consumed as part of our data warehouse code. Combined, these factors mean that changing the field would require a significant coordinated change across multiple code bases and probably downtime so it's not going to happen.

Also subsequently, @jouvin has started to advocate having the broker populate more items in the templates, thus further negating (2) above. In order for this approach to work, the broker and templates must be aligned to avoid divergent code paths that are time consuming to maintain and cross-communication issues between the primary broker developers and community users.

I am cognisant that in encouraging the change in #142 I made a significant error in judgement for which I apologise. I believe that at present only 1 site is effected, @jrha since the three others sites using Aquilon are on more recent versions of Aquilon where the prior structures are now required.

@jrha : how painful would it be for you to back out of this and use an alternative structure?

14.5.0-rc6 issues

I just gave a try to 14.5.0-rc6. Unfortunately the issue with component templates not being updated is still there (this is still the 14.4.0-rc3 version). Something wrong in the release script.
And there is a minor problem in version.pan also caused by the release script.

component authconfig : krb5_realm = ARRAY(0x39a5830)

When I deploy my sssd config with the option '"krb5_realm", list(LAPP.IN2P3.fr)"".
I obtain this error in /etc/sssd/sssd.conf :
krb5_realm = ARRAY(0x39a5830)

In the authconfig schema, if I replace "krb5_realm" ? string[] by "krb5_realm" ? string, all its ok.

New include syntax not working in with debug

Hi,

I tried to build a scdb-vanilla from this repo but this commit (d60221f) introduced some syntax error :

[panc-check-syntax] parse error [/tmp/scdb-vanilla/cfg/quattor/dev/quattor/aii/ks/config.pan:405.112-405.405]
[panc-check-syntax] 
[panc-check-syntax] Encountered: AII_KS_OS_MAJOR_SPECIFIC_INCLUDE

Problematic lines are :

include debug('KS specific configuration for OS major version: '+to_string(AII_KS_OS_MAJOR_SPECIFIC_INCLUDE)); AII_KS_OS_MAJOR_SPECIFIC_INCLUDE;

include debug('KS specific configuration for OS minor release: '+to_string(AII_KS_OS_MINOR_SPECIFIC_INCLUDE)); AII_KS_OS_MINOR_SPECIFIC_INCLUDE;

Removing the debug or using curly-braces fixes the issue.

I'm wondering how this pull request has been merged without any check (build or check syntax).
I suggest every new pull request should build the scdb-vanilla script (from scdb repo) and block the merge in case of compilation issue.

Aquilon schema cleanup

A few (minor) changes are needed in Aquilon schemas after the discussion in quattor/aquilon#90 and the related PRs:

  • optional room property in structure_rack is no longer used by the broker and should be removed.
  • name property in structure_archetype should be made mandatory once quattor/aquilon#89 is merged

Quattor 13.9 definition

Hi,
I'm working on the move to 13.9 (Yum-based)
I have some issues with some provided templates (template-library-core/quattor/client/rpms.pan) : they are written for 13.1

I know spma and rpmt-py are no longer used so they are not on the 13.9 repository.
Do I need to fix dependencies in templates or someone has an up to date version to work with 13.9 out of the box ?

Cheers

14.5.0-rc8: perl-AppConfig-caf vs perl-AppConfig

Trying to update one machine to 14.5.0-rc8 from 14.4.0-rcX, I had some conflicts. I resolved them by manually removing perl-AppConfig-caf (this broke quattor) and installing perl-AppConfig.

2014/06/05-12:49:39 -----------------------------------------------------------
2014/06/05-12:49:54 [WARN] running transaction produced warnings: Error: Transaction Check Error:
  file /usr/share/man/man3/AppConfig::Args.3pm.gz from install of perl-AppConfig-1.66-6.el6.noarch conflicts with file from package perl-AppConfig-caf-1.9.1-1.noarch
  file /usr/share/man/man3/AppConfig::File.3pm.gz from install of perl-AppConfig-1.66-6.el6.noarch conflicts with file from package perl-AppConfig-caf-1.9.1-1.noarch
  file /usr/share/man/man3/AppConfig::Getopt.3pm.gz from install of perl-AppConfig-1.66-6.el6.noarch conflicts with file from package perl-AppConfig-caf-1.9.1-1.noarch
  file /usr/share/man/man3/AppConfig::State.3pm.gz from install of perl-AppConfig-1.66-6.el6.noarch conflicts with file from package perl-AppConfig-caf-1.9.1-1.noarch
  file /usr/share/man/man3/AppConfig::Sys.3pm.gz from install of perl-AppConfig-1.66-6.el6.noarch conflicts with file from package perl-AppConfig-caf-1.9.1-1.noarch

    Error Summary
-------------


2014/06/05-12:49:54 [WARN] Command output: Loaded plugins: priorities, versionlock
Setting up Yum Shell
> Setting up Install Process
Package libstdc++-4.4.7-3.el6.i686 already installed and latest version
Package compat-libstdc++-33-3.2.3-69.el6.i686 already installed and latest version
> --> Running transaction check
---> Package perl-AppConfig.noarch 0:1.66-6.el6 will be installed
--> Finished Dependency Resolution
--> Finding unneeded leftover dependencies
Found and removing 0 unneeded dependencies
Success resolving dependencies
> --> Running transaction check
--> Finished Dependency Resolution
--> Finding unneeded leftover dependencies
Found and removing 0 unneeded dependencies

================================================================================
 Package              Arch         Version             Repository          Size
================================================================================
Installing:
 perl-AppConfig       noarch       1.66-6.el6          sl640_x86_64        86 k

Transaction Summary
================================================================================
Install       1 Package(s)

Total download size: 86 k
Installed size: 223 k
Downloading Packages:
Running rpm_check_debug
Running Transaction Test
> Leaving Shell

2014/06/05-12:49:54 [ERROR] Failed running transaction: Error: Transaction Check Error:
  file /usr/share/man/man3/AppConfig::Args.3pm.gz from install of perl-AppConfig-1.66-6.el6.noarch conflicts with file from package perl-AppConfig-caf-1.9.1-1.noarch
  file /usr/share/man/man3/AppConfig::File.3pm.gz from install of perl-AppConfig-1.66-6.el6.noarch conflicts with file from package perl-AppConfig-caf-1.9.1-1.noarch
  file /usr/share/man/man3/AppConfig::Getopt.3pm.gz from install of perl-AppConfig-1.66-6.el6.noarch conflicts with file from package perl-AppConfig-caf-1.9.1-1.noarch
  file /usr/share/man/man3/AppConfig::State.3pm.gz from install of perl-AppConfig-1.66-6.el6.noarch conflicts with file from package perl-AppConfig-caf-1.9.1-1.noarch
  file /usr/share/man/man3/AppConfig::Sys.3pm.gz from install of perl-AppConfig-1.66-6.el6.noarch conflicts with file from package perl-AppConfig-caf-1.9.1-1.noarch

Error Summary
-------------


2014/06/05-12:49:54 [INFO] configure on component spma executed, 1 errors, 2 warnings

unittest location

We need a subdirectory to put unittests in that will not be packaged/part of the release

is_fqdn function also matches partially qualified hostnames

The is_fqdn function in types.pan will match partially qualified hostnames as well as proper FQDNs.
For instance, it will match example.org (which it should) but will also match example.
The regex will match a properly formed hostname component followed by a literal . or the end of the string.

This means there is no functionality available to ensure that a value is actually an FQDN as opposed to a hostname.

Furthermore, the is_hostname function seems to rely on this behaviour (it should match IPs, FQDNs and partially qualified hostnames) and would be (partially) broken by a fix to is_fqdn.

functions/package: change values for PKG_VERSION_xxx constants

Current values are rather misleading/counter-intuitive. And they don't allow to do a numeric comarison like:

if ( pkg_compare_version('x', '1.6.0') <= PKG_VERSION_EQUAL ) ...

to test if a version is less or equal.

This should not be a problem to define less=-1 and greater=1 when all the PRs related to pkg_compare_version() have been merged. We just need to tell site to update their local templates if they use this function.

Tag this repo as part of the release process after updating it

Hi,

Here is some suggestions, after some private discussions, on how to maintain this repo as part of the release process. The main goal of this repo is to provide a central point for distribution all the templates related to a Quattor release, avoiding that each site has to go to each invidividual repo to grab all the necessary information. In addition it provides some manually maintained templates (not imported from somewhere else) that are required to define a profile schema, define a few functions or configure the Quattor client/server.

Based on my recent experience trying to update the repo manually, here are the actions that should be done by the release script if we wanted to maintain automatically this repo contents:

  • After releasing configuration-modules-core/grid, update the component templates. This can be done easily with updateComponents script, in SCDB utils/misc (nothing SCDB specific). This is what I used.
  • After releasing AII, update the AII-related templates. This can be done easily with updateAII script, in SCDB utils/misc (nothing SCDB specific). This is what I used.
  • Update the Quattor version in quattor/client/version.pan: this is probably the most tricky part as the repo structure is not really appropriate to maven variable processing. May be something we should change in the future with an appropriate src and target structure.

SCDB scripts can be found in scdb repo.

Cheers,

Michel

Figure out proper location for profiles build information

Create a schema entry that can hold profile build related metadata.

E.g. in the hypothetical example /system/quattorid, with quattorid a string, one can then inject this data via the panc initial data options:

  • add this to the templates with default value
"/system/quattorid" ?= 'DUMMY_SOMETHING_WENT_WRONG';
  • come up with data, e.g.
# Sets the deployment identifier based on the date of the current
# commit.
set_quattorid() {
    cid=`git log -1  --date=raw | awk '/Date:/{print $2}'`
    branch=`git name-rev HEAD|awk '{print $2}'`
    if [ "$?" != "0" ]
    then
        echo "Something went wrong with git log."
        exit 1
    fi

    echo "$branch-$cid"
}
id="nlist('system', nlist('quattorid', '`set_quattorid`'))"
  • to use the data, add on the ant command line
    • if you use ant (e.g. SCDB): -Dpan.root.element="$id"
    • pure panc: --initial-data "$id"

Change type legacy_binary_affirmation_string to legacy_yes_or_no_string

As stated in PR quattor/configuration-modules-core#816 ( https://github.com/quattor/configuration-modules-core/pull/816/files#r71087664), legacy_binary_affirmation_string does not make obvious what are the valid values of this type. Renaming this type to legacy_yes_or_no_string may be better and avoid having to look at the type definition to know the valid values.

Making the change involves modifying the configuration modules (core only?) using it. See quattor/configuration-modules-core#749.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.