How to generate text without fintetune?

[minimaxir]

How to generate text without fintetune?

[woctezuma]

First, run:

import gpt_2_simple as gpt2

gpt2.download_gpt2(model_name='345M')

Then:

!mkdir -p checkpoint/
!cp -r models/345M checkpoint/run1

Finally:

import gpt_2_simple as gpt2

sess = gpt2.start_tf_sess()
gpt2.load_gpt2(sess)

gpt2.generate(sess)

[woctezuma]

With version 0.6, you should be able to directly run:

import gpt_2_simple as gpt2

model_name='774M'

gpt2.download_gpt2(model_name=model_name)

sess = gpt2.start_tf_sess()
gpt2.load_gpt2(sess, model_name=model_name)

gpt2.generate(sess, model_name=model_name)

[woctezuma]

True. It seems that there is a bit of redundancy w.r.t. parameters being loaded by both load_gpt2 and generate. However, by looking at the code, the generate function seems to require the model name to be able to encode the prefix. Paging @minimaxir, because he knows better than me.

def load_gpt2(sess,
              [...]
              multi_gpu=False):
    """Loads the model checkpoint or existing model into a TensorFlow session
    for repeated predictions.
    """
   [...]

    hparams = model.default_hparams()
    with open(os.path.join(checkpoint_path, 'hparams.json')) as f:
        hparams.override_from_dict(json.load(f))

   [...]

def generate(sess,
             [...]
             include_prefix=True):
    """Generates text from a model loaded into memory.
    Adapted from:
    https://github.com/openai/gpt-2/blob/master/src/interactive_conditional_samples.py
    """
   [...]

    enc = encoder.get_encoder(checkpoint_path)
    hparams = model.default_hparams()
    with open(os.path.join(checkpoint_path, 'hparams.json')) as f:
        hparams.override_from_dict(json.load(f))

    if prefix:
        context = tf.compat.v1.placeholder(tf.int32, [batch_size, None])
        context_tokens = enc.encode(prefix)

   [...]

[chrisrytting]

I'm confused @woctezuma . Why do you have to indicate model_name to the generate method? Shouldn't gpt2 contain the parameters once you load it in your penultimate line?

[avinregmi]

@minimaxir

[jielyugt]

The redundancy is confusing. What happens if I load a checkpoint trained on 124M for load_gpt2(), and load 124M without any fine-tuning for generate()?

[chrisrytting]

Yeah this seems weird to me too because if @woctezuma is right, then you hypothetically could use different checkpoints of the same model and not have any errors thrown. We'll see if @minimaxir can shed some light.