Including a YAML file within another

I have written an extension to the PyYAML Loader class that allows you to include/import YAML files from within another YAML file.

I’m currently writing a program that will data from a YAML file and create a CV using a template engine. To aid flexibility, I wanted to allow the user to be able to include separate YAML files (e.g. education.yaml, skills.yaml, etc) within a master YAML file which would then be used to make the CV.

An initial search found this, which allows users to include a YAML file from within another. However, it’s quite a limited solution – it assumes you only want to include a single file. For my CV program, I wanted to be able to include multiple files within a single section. This could let you have separate files containing publications according to subject, and then include different groups of publications for different CVs.

My solution now supports all three kinds of YAML node: scalar, sequence and mapping.

import yaml
import os.path

class Loader(yaml.Loader):
    def __init__(self, stream):
        self._root = os.path.split(stream.name)[0]
        super(Loader, self).__init__(stream)
        Loader.add_constructor('!include', Loader.include)
        Loader.add_constructor('!import',  Loader.include)

    def include(self, node):
        if   isinstance(node, yaml.ScalarNode):
            return self.extractFile(self.construct_scalar(node))

        elif isinstance(node, yaml.SequenceNode):
            result = []
            for filename in self.construct_sequence(node):
                result += self.extractFile(filename)
            return result

        elif isinstance(node, yaml.MappingNode):
            result = {}
            for k,v in self.construct_mapping(node).iteritems():
                result[k] = self.extractFile(v)
            return result

        else:
            print "Error:: unrecognised node type in !include statement"
            raise yaml.constructor.ConstructorError

    def extractFile(self, filename):
        filepath = os.path.join(self._root, filename)
        with open(filepath, 'r') as f:
            return yaml.load(f, Loader)

Please note that the arguements to !include/!import must be on the same line (due to PyYAML parser). This allows you to specify YAML files like:

education:    !include education.yaml

activities:   !include [schools.yaml, conferences.yaml, workshops.yaml]

publications: !include {peer_reviewed: publications/peer_reviewed.yaml, internal: publications/internal.yaml}
Advertisement

5 thoughts on “Including a YAML file within another

  1. Carsten Lemmen

    Dear David,

    thanks for the Loader class. I want to include it in our open source software https://sf.net/p/mossco, which is GPL-licensed. How about your code snippet? Any thoughts on licensing issues from your side? Is it public domain, or CC-by?

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s