Python: Missing arguments in string formatting (lazy eval)

Question

i'm trying to do some "post"/"lazy" evaluation of arguments on my strings. Suppose i've this:

s = "SELECT * FROM {table_name} WHERE {condition}"

I'd like to return the string with the {table_name} replaced, but not the {condition}, so, something like this:

s1 = s.format(table_name = "users")

So, I can build the hole string later, like:

final = s1.format(condition= "user.id = {id}".format(id=2))

The result should be, of course:

"SELECT * FROM users WHERE user.id = 2"

I've found this previous answer, this is exactly what I need, but i'd like to use the format string function.

python, format string Thank you for your help!

Naturally, you're going to escape all the replacement strings to prevent an SQL injection attack - right? ;) — Li-aung Yip, Mar 31 '12 at 12:42
I hope you're not *ever* letting *any* user input get into this? This sort of code is asking for security holes. — Chris Morgan, Mar 31 '12 at 12:42
Of course i won't do it fellas, maybe it wasn't the best example, but was what first came to my mind. — santiagobasulto, Mar 31 '12 at 12:56

score 7 · Answer 1 · answered Mar 31 '12 at 12:15

7

You can replace the condition with itself:

s.format(table_name='users', condition='{condition}')

which gives us:

SELECT * FROM users WHERE {condition}

You can use this string later to fill in the condition.

answered Mar 31 '12 at 12:15

Simeon Visser

118,920
18
185
180

replacing something with itself doesnt seem very Pythonic at least from my understanding... – jamylak Mar 31 '12 at 12:43
Well, it's short, clear and it works so that's Pythonic to me :) – Simeon Visser Mar 31 '12 at 12:47

Karoly Horvath · Accepted Answer · 2012-03-31T12:47:11.587

You can't use the format function because it will raise a KeyError.

string.Template supports safe substituion:

from string import Template
s = Template('SELECT * FROM $table_name WHERE $condition')
s.safe_substitute(table_name='users')

'SELECT * FROM users WHERE $condition'

If you use plain variable names (no format specifiers, no indexing, etc..) this will also work (thanks @Simeon Visser for the idea):

def myformat(s, *args, **kwargs):
  while True:
    try:
      return s.format(*args, **kwargs)
    except KeyError as e:
      e=e.args[0]
      kwargs[e] = "{%s}" % e

s = "SELECT * FROM {table_name} WHERE {condition}" 
myformat(s, table_name="users")

'SELECT * FROM users WHERE {condition}'

score 1 · Answer 3 · answered Sep 30 '13 at 19:55

This builds on @Karoly Horvath's answer to add support for index keys and attribute access on named keys:

import re

def my_format(template, *args, **kwargs):
  next_index = len(args)
  while True:
    try:
      return template.format(*args, **kwargs)
    except KeyError as e:
      key = e.args[0]
      finder = '\{' + key + '.*?\}'
      template = re.sub(finder, '{\g<0>}', template)
    except IndexError as e:
      args = args + ('{' + str(next_index) + '}',)
      next_index += 1

So to test it out:

class MyObj:
  bar = 'baz'

  def __repr__(self):
    return '<MyObj instance>'

my_obj = MyObj()

template = '{0}, {1}, {foo}, {foo.bar}, {0}, {10}, {missing}'
print my_format(template)
print my_format(template, '1st', '2nd', missing='Not Missing')
print my_format(template, foo=my_obj)

Output:

{0}, {1}, {foo}, {foo.bar}, {0}, {10}, {missing}
1st, 2nd, {foo}, {foo.bar}, 1st, {10}, Not Missing
{0}, {1}, <MyObj instance>, baz, {0}, {10}, {missing}

score 1 · Answer 4 · edited Jul 03 '17 at 07:50

I have been using this function for some time now, which casts the Dict of inputted keyword arguments as a SafeDict object that subclasses Dict.

def safeformat(str, **kwargs):
        class SafeDict(dict):
                def __missing__(self, key):
                    return '{' + key + '}'
        replacements = SafeDict(**kwargs)
        return str.format_map(replacements)

I didn't make this up, but I think it's a good solution. The one downside is that you can't call mystring.safeformat(**kwargs) - of course, you have to call safeformat(mystring,**kwargs).

If you're really interested in being able to call mystr.safeformat(**kwargs) (which I am interested in doing!), consider using this:

class safestr(str):
    def safeformat(self, **kwargs):
        class SafeDict(dict):
                def __missing__(self, key):
                        return '{' + key + '}'
        replacements = SafeDict(**kwargs)
        return safestr(self.format_map(replacements))

You can then create a safestr object as a = safestr(mystr) (for some str called mystr), and you can in fact call mystr.safeformat(**kwargs).

e.g.

mysafestr = safestr('Hey, {friendname}. I am {myname}.')
print(mysafestr.safeformat(friendname='Bill'))

prints

Hey, Bill. I am {myname}.

This is cool in some ways - you can pass around a partially-formatted safestr, and could call safeformat in different contexts. I especially like to call mystr.format(**locals()) to format with the appropriate namespace variables; the safeformat method is especially useful in this case, because I don't always carefully looks through my namespace.

The main issue with this is that inherited methods from str return a str object, not a safestr. So mysafestr.lower().safeformat(**kwargs) fails. Of course you could cast as a safestr when using safeformat:

safestr(mysafestr.lower()).safeformat(**kwargs),

but that's less than ideal looking. I wish Python just gave the str class a safeformat method of some kind.

score 0 · Answer 5 · answered Nov 12 '14 at 19:14

This is a slight change to @ShawnFumo's answer which has a small bug. We need to add a word boundary check (the \b in the regular expression) to ensure that we are matching only the failing key and another key that starts with the same string. This prevents a missing {foo} key from also treating {food} and {foolish} as if they were missing.

import re

def my_format(template, *args, **kwargs):
  next_index = len(args)
  while True:
    try:
      return template.format(*args, **kwargs)
    except KeyError as e:
      key = e.args[0]
      finder = r'\{' + key + r'\b.*?\}'
      template = re.sub(finder, r'{\g<0>}', template)
    except IndexError as e:
      args = args + ('{' + str(next_index) + '}',)
      next_index += 1

So to test it out:

class MyObj:
  bar = 'baz'

  def __repr__(self):
    return '<MyObj instance>'

my_obj = MyObj()

template = '{0}, {1}, {foo}, {foo.bar}, {0}, {10}, {missing}'
print my_format(template)
print my_format(template, '1st', '2nd', missing='Not Missing')
print my_format(template, foo=my_obj)

print

template2 = '{foo} and {food}'
print my_format(template2)
print my_format(template2, food='burger')
print my_format(template2, foo=my_obj, food='burger')

Output:

{0}, {1}, {foo}, {foo.bar}, {0}, {10}, {missing}
1st, 2nd, {foo}, {foo.bar}, 1st, {10}, Not Missing
{0}, {1}, <MyObj instance>, baz, {0}, {10}, {missing}

{foo} and {food}
{foo} and burger
repr(<MyObj instance>) and burger

score 0 · Answer 6 · answered Oct 10 '15 at 17:04

An alternative to string.Template.safe_substitute could be subclassing string.Formatter like so:

class LazyFormatter(string.Formatter):
    def get_value(self, key, args, kwargs):
        '''Overrides string.Formatter.get_value'''
        if isinstance(key, (int, long)):
            return args[key]
        else:
            return kwargs.get(key, '{{{0}}}'.format(key))

lazyfmt = LazyFormatter()
print lazyfmt.format("{field}: {value}", **{'field': 'foo'})

Output:

foo: {value}

Python: Missing arguments in string formatting (lazy eval)

6 Answers6

Linked