Skip to main content

Question Type: this_or_that_question

What are the trade offs?

Updated over 3 weeks ago

When and Why to Use

Use this to ask binary forced-choice questions across multiple comparisons. It's ideal for:

  • Preference testing between paired items

  • Quick-fire brand or concept comparisons

  • Engaging users with fast-paced choice tasks

Supports randomization and media-based comparisons.

Portrait Experience

  • Each pair appears as a row with two side-by-side buttons or tiles

  • Users select one item at a time

Landscape Experience

  • All pairs displayed in grid format

  • Responsive to keyboard or remote selection

  • Layout adapts to show multiple pairs simultaneously

Configuration Options

Option

Type

Required

Default

Description

question

string

yes

-

The main prompt for the task

row_options

`List[List[str

MediaItem]]`

yes

-

randomize

bool

no

False

Shuffle the order of rows

randomize_columns

bool

no

False

Shuffle left/right order within each pair

default

Dict[str, str]

no

random choices

Default test responses per pair

recodes

Dict[str, str]

no

-

Map raw values to grouped outputs

image

MediaItem

no

-

Top-level image shown above the task

image_label_field

str

no

-

Label to show above any media items

show_image_label

bool

no

True

Whether to show labels on image tiles

image_size

Tuple[int, int]

no

-

Size of displayed images

custom_validator

`Callable[[Dict[str, str]], str

None]`

no

-

**topics

dict

no

-

Used for token substitution and reporting

Example Code

s.this_or_that_question( "Which of the following do you prefer?", row_options=[ ["Apples", "Oranges"], ["Coke", "Pepsi"], ["Dogs", "Cats"], ["Winter", "Summer"] ] )

With recodes:

s.this_or_that_question( "Which of the following do you prefer?", row_options=[["Netflix", "Disney+"], ["Prime", "Apple TV"]], recodes={ "Netflix": "streaming", "Disney+": "streaming", "Prime": "ecommerce", "Apple TV": "streaming" } )

With custom validator:

s.this_or_that_question( "Choose your favorite in each pair", row_options=[["Day", "Night"], ["Hot", "Cold"]], custom_validator=lambda d: "Please vary your responses" if len(set(d.values())) == 1 else None )

Notes

  • Each row_options entry must contain exactly two items

  • Randomizing column order helps reduce left/right bias

  • recodes simplify grouped reporting or segmentation

  • Use custom_validator to enforce answer patterns or detect straight-lining

Did this answer your question?