Testing the editor bundle with test data

When you upload an editor bundle, the optional test_data parameter gives Cloud Services a document to test the editor with. Cloud Services uses the bundle to convert the documents of your users. A bundle that changes content during this conversion causes data loss, and the check with test_data finds such problems before your users do.

This guide explains how the check works, how to prepare the test data, and how to find the cause of a failed check. The examples use the editor of the Vite example project, but the guide applies to bundles from any bundler.

Cloud Services checks the uploaded bundle in a browser on the server:

1. It creates the editor from the bundle, with the uploaded configuration and with test_data as the content.
2. It takes the operations that the first editor made when it loaded the test data, and applies them to a second, empty editor. Cloud Services builds the documents of collaboration sessions from operations in the same way.
3. It compares the content of the second editor, from editor.getData( { trim: 'none' } ), with test_data. The comparison is exact: every character, space, and entity must be the same.

If the content is different, the upload fails with HTTP 409. The response contains these fields, among others:

{
	"message": "Converted data result is not the same as provided in \"test_data\".",
	"explanation": "Editor changed input data",
	"action": "Check document. Headers are converted to paragraph+header+paragraph structure, (…)",
	"data": {
		"convertedData": "<h2>A heading</h2><p>A paragraph.</p>"
	}
}

The data.convertedData field shows what the editor makes of your test data. In this example, the test data had <h1>A heading</h1>. Compare the two strings to find the change. The action field has the same general hints for each failed check. The Changes that the editor makes section explains these hints and lists the most common changes.

Thus the test data must be exactly what the editor returns. Test data that you write by hand almost always fails.

The test_data parameter is optional, but an upload without it checks much less:

• For a single-root editor, and for a multi-root editor without rootsAttributes in the configuration, Cloud Services only makes sure that the editor starts. Nothing compares content.
• For a multi-root editor with rootsAttributes, Cloud Services uses <p>Test</p> as the content of each root of rootsAttributes. Then it compares the output of the editor with this content.

In both cases, the response has a warning:

Missing "testData" field. It is highly recommended to test the uploaded editor using real content. (…)

Make the test data from the output of your editor:

1. Keep sample documents in your repository, for example in test-data/samples/. Use real documents of your application, and documents that you write for the checklist below.

2. Open each sample in the editor of your application, with the same plugins and configuration as the bundle.

3. Take the content with getData( { trim: 'none' } ), the same call that Cloud Services uses. For example, run this command in the developer tools of the browser:

   copy( document.querySelector( '.ck-editor__editable' ).ckeditorInstance.getData( { trim: 'none' } ) );

   Copy code

   The command gets the editor instance from the editable element with the ckeditorInstance property. CKEditor 5 has no global list of editor instances.

4. Compare the content with the sample, for example with the local check. Make sure that each change is correct for your application. If a change loses content, fix the plugin or the configuration first, as in the example with a custom plugin.

5. Save the content without changes, for example in test-data/document.html. If your upload script reads the test data from a file, it must remove the final new line that code editors add.

6. Make new test data each time you update CKEditor 5, add or change a plugin, or change the configuration. For example, since v46.0.0, the editor writes the data-list-item-id attribute in list items. Thus test data with lists from v45 or older fails after the update.

The upload does not find the changes of step 4. The test data is the output of the editor, so it already contains them. The Changes that the editor makes section explains the common changes.

The trim: 'none' option matters only for a document without content, for example a document with only empty paragraphs. Then editor.getData() returns an empty string, but Cloud Services compares the content, for example <p>&nbsp;</p>. An empty string is not valid test data: the upload fails with HTTP 400 and the Editor test data must be at least 1 characters message.

Put in content that uses every part of your editor:

• Every feature of the editor: headings, lists, tables, images, links, and the other plugins in builtinPlugins.
• Every element and attribute that your custom plugins create or read. Put the old formats that your application still stores in the samples, as in the example with a custom plugin. The editor writes the current format, so test data with an old format always fails.
• Empty elements, for example an empty paragraph and an empty table cell.
• White space: spaces next to formatting and inline widgets, and soft line breaks.
• Special characters and entities: &, <, >, quotes, non-breaking spaces, and symbols such as ©.
• Text in other scripts: for example Chinese, Arabic and other right-to-left text, and emoji.
• Comments and suggestions, if your editor has collaboration features. Cloud Services accepts their markers in the test data.
• Real documents of your application, for example the largest and the most complex ones.

The upload request, with the bundle, the configuration, and the test data, must be smaller than 10 MB.

When the check fails, compare data.convertedData with your test data. The sections below describe the common changes. The results come from uploads with the editor of the Vite example project. Your plugins, your configuration, and your version of CKEditor 5 can give other results, but data.convertedData always shows the result for your editor.

An empty block gets a filler, so that it stays visible: <p></p> and <p> </p> become <p>&nbsp;</p>. A space next to an inline widget also becomes  . For example, Dear <span class="placeholder">… becomes Dear&nbsp;<span class="placeholder">….

The editor collapses white space, as HTML does. Two or more spaces in a row become one space. The editor also removes the spaces at the start and the end of a block, so <p> Leading and trailing </p> becomes <p>Leading and trailing</p>. A new line between <p>One</p> and <p>Two</p> disappears too.

These inputs need no change: <p>&nbsp;</p>, <br> in a paragraph, and   in text.

The editor writes most characters as characters. Named entities such as © and € become © and €. In text, "quotes" and ' become "quotes" and '. Only &, <, >, and   stay entities in text. In attribute values, " also stays.

Emoji and text in other scripts, for example Chinese and Arabic, need no change.

The plugins of the editor and their options decide which elements and attributes the output has. The heading options set the levels of the headings. The default options have the levels <h2>, <h3>, and <h4>, so <h1> becomes <h2>, and <h5> and <h6> become <p>. Each format has one element, so <b> and <em> become <strong> and <i>.

The editor removes the elements and attributes that no plugin supports, but it keeps their text. Without a strikethrough plugin, <s> disappears. Without a plugin for the dir attribute, <p dir="rtl"> becomes <p>. General HTML Support keeps only the elements and attributes that its configuration allows, so <article> and style="…" can disappear too. The link feature keeps only href, unless you configure link decorators: <a target="_blank" rel="noopener" href="…"> becomes <a href="…">.

The editor writes tables, lists, and images in its own structure. It wraps a table in <figure class="table">, adds <tbody>, and fills an empty cell. For example, <table><tr><td>A</td><td></td></tr></table> becomes <figure class="table"><table><tbody><tr><td>A</td><td>&nbsp;</td></tr></tbody></table></figure>. A table in <figure class="table"> with <thead> and <th> needs no change.

The editor gives list items random IDs: <ul><li>One</li></ul> becomes <ul><li data-list-item-id="e…">One</li></ul>. Thus take lists from the output of the editor.

An image without <figure class="image"> becomes an inline image. Outside a paragraph, <img src="…" alt="…"> becomes <p><img src="…" alt="…"></p>. An image in a paragraph needs no change.

The HTML parser of the browser fixes invalid HTML before the editor reads it. A heading cannot be in a paragraph, so the parser splits the paragraph: <p>Before <h2>Heading</h2> after</p> becomes <p>Before</p><h2>Heading</h2><p>after</p><p>&nbsp;</p>. This is the “paragraph+header+paragraph” structure that the action field of the response mentions.

Some plugins change the structure of the document. With the Title plugin, the first block becomes the title, and the body gets an empty paragraph: <p>Body only</p> becomes <h1>Body only</h1><p>&nbsp;</p>. Comment and suggestion markers need no change.

The check uses the uploaded configuration, so the test data must match it:

• The editor removes the content that needs a plugin from removePlugins. For example, with GeneralHtmlSupport in removePlugins, <section class="note"><p>A note.</p></section> becomes <p>A note.</p>. Remove only plugins that do not change the content.
• Cloud Services also removes each plugin that requires a plugin from removePlugins, if builtinPlugins lists the removed plugin. It does this without an error, so the content of these plugins disappears too. For example, if a plugin that converts <s> requires a UI plugin from removePlugins, <s>Struck text</s> becomes Struck text. Only the test data shows this problem. If builtinPlugins does not list the removed plugin, the upload stops with the plugincollection-required error.
• The upload does not check the names in removePlugins. A name with a typo removes nothing, and the plugin stays in the editor.
• The options that change content, for example the heading options, must be the same as in your application.
• A name in plugins or extraPlugins that the bundle does not have stops the upload with HTTP 409 and the plugincollection-plugin-not-found {"plugin":"…"} error.

For a multi-root editor, set is_multi_root_editor to true in the upload request. Then test_data is a JSON string. Its keys are the names of the roots, and its values are the content of each root:

{"header":"<p>Header</p>","content":"<p>Content</p>"}

Keep these rules for the multi-root test data:

• The test data must contain each root of the editor. With the legacy rootsAttributes option, a missing root fails the upload with HTTP 400 and the Invalid test_data for given multi-root editor bundle. message. More roots and another order of the roots are allowed.
• Since CKEditor 5 v48, the root attributes are in the roots.<rootName>.modelAttributes option. With this option, the editor creates each root of the configuration. Then a missing root fails the upload with HTTP 409 and the Converted data result is not the same as provided in "test_data". message. The data.convertedData field of the response contains the missing root.
• Cloud Services compares the test data with the output of the editor after JSON.stringify(). Thus make the JSON with JSON.stringify(). Formatted JSON fails with “Editor changed input data”. JSON from other encoders can also fail, because they can add spaces or write other escapes, for example \u00e9 for é.
• If the configuration has rootsAttributes, invalid JSON fails with HTTP 500 and the Internal server error. message. Without rootsAttributes, it fails with HTTP 409 and a message that starts with Broken editor build. Details: Provided data cannot be applied to a multi-root editor correctly. Make sure that the test data is valid JSON before the upload.

To take the test data from a multi-root editor, use the getFullData() method:

const testData = JSON.stringify( editor.getFullData( { trim: 'none' } ) );

The upload is the final check, but a local check finds problems faster. The following module runs the same check in your application: it starts the editor with the test data and compares the output. Add it to your application, for example as src/check-test-data.js. Change the imports to the editor and the configuration of your application:

// src/check-test-data.js

import { AppEditor } from './editor/editor.js';
import { editorConfig, pluginsRemovedOnServer } from './editor/config.js';

// The plugins of this editor that Cloud Services removes itself. The list depends on the version of Cloud Services,
// for example of an on-premises installation. The `GET /editors/{bundle_version}/config` request shows the full list.
const pluginsRemovedByCloudServices = [
	'Autosave',
	'PresenceList',
	'RealTimeCollaborativeComments',
	'RealTimeCollaborativeEditing',
	'RealTimeCollaborativeRevisionHistory',
	'RealTimeCollaborativeTrackChanges',
	'RevisionHistory'
];

// The comment and suggestion markers in the format that the editor writes. Cloud Services accepts them in `test_data`,
// but this check has no comment and suggestion data, so it removes the markers first.
const MARKER_ELEMENTS = /<(comment|suggestion)-(start|end)\b[^>]*><\/\1-\2>/g;
const MARKER_ATTRIBUTES = / data-(comment|suggestion)-(start|end)-(before|after)="[^"]*"/g;

// Starts the editor with the test data, as Cloud Services does during the upload, and compares the output.
// If the output is different, it also checks each top-level element, to find the elements that change.
export async function checkTestData( testDataWithMarkers ) {
	const testData = testDataWithMarkers.replace( MARKER_ELEMENTS, '' ).replace( MARKER_ATTRIBUTES, '' );
	const element = document.createElement( 'div' );

	document.body.appendChild( element );

	const editor = await AppEditor.create( {
		...editorConfig,
		attachTo: element,
		licenseKey: import.meta.env.VITE_CKEDITOR_LICENSE_KEY,
		removePlugins: [ ...pluginsRemovedOnServer, ...pluginsRemovedByCloudServices ],
		root: {
			initialData: testData
		}
	} );

	const output = editor.getData( { trim: 'none' } );
	const changes = [];
	let firstDifference = null;

	if ( output !== testData ) {
		// The browser parses the test data before the check divides it, so a change of entities or other details
		// of the HTML source shows only here, at the first character where the output is different.
		let index = 0;

		while ( testData[ index ] === output[ index ] ) {
			index++;
		}

		const start = Math.max( 0, index - 20 );

		firstDifference = { input: testData.slice( start, index + 40 ), output: output.slice( start, index + 40 ) };

		const template = document.createElement( 'template' );

		template.innerHTML = testData;

		for ( const node of template.content.childNodes ) {
			// White space and comments between the elements are not content. The first difference shows their changes.
			if ( node.nodeType === Node.COMMENT_NODE || ( node.nodeType === Node.TEXT_NODE && !node.textContent.trim() ) ) {
				continue;
			}

			// `outerHTML` exists only for elements. The wrapper also gives the HTML of text.
			const wrapper = document.createElement( 'div' );

			wrapper.append( node.cloneNode( true ) );

			const input = wrapper.innerHTML;

			editor.setData( input );

			const elementOutput = editor.getData( { trim: 'none' } );

			if ( elementOutput !== input ) {
				changes.push( { input, output: elementOutput } );
			}
		}
	}

	await editor.destroy();
	element.remove();

	return { passes: output === testData, output, firstDifference, changes };
}

If the output is different, the result shows where:

• firstDifference contains the test data and the output near the first character that is different.
• changes contains each top-level element that changes when the editor loads it alone, with the output of the editor.

The changes list can be empty, for example when only an entity changes. The browser parses the test data before the module divides it into elements, so the elements already have the format of the browser. The list also skips white space and comments between the elements. In these cases, firstDifference shows the change.

With the Title plugin, changes lists every element, also the correct ones. When the editor loads one element alone, it makes this element the title and adds an empty paragraph for the body. In this case, firstDifference shows the real change.

A test page checks the samples and the test data with this module. Add the page to your application, for example as test-data.html:

<!-- test-data.html -->

<!doctype html>
<html lang="en">
	<head>
		<meta charset="UTF-8" />
		<title>Test data check</title>
	</head>
	<body>
		<pre id="result">Checking the test data…</pre>
		<script type="module" src="/src/test-data-page.js"></script>
	</body>
</html>

The src/test-data-page.js module of the page reads the samples from test-data/samples/ and the test data from test-data/document.html. It uses the ?raw import and the import.meta.glob() function of Vite. With another bundler, use its method to import files as text.

// src/test-data-page.js

import testData from '../test-data/document.html?raw';
import { checkTestData } from './check-test-data.js';

const samples = import.meta.glob( '../test-data/samples/*.html', { query: '?raw', import: 'default', eager: true } );
const report = [];

// Checks one document and describes the result.
async function check( name, data ) {
	// Cloud Services compares the data exactly, so remove the final new line of the file, as the upload script does.
	const { passes, firstDifference, changes } = await checkTestData( data.trimEnd() );

	return passes ? `${ name }: no changes.` : `${ name }: changes:\n${ JSON.stringify( { firstDifference, changes }, null, 2 ) }`;
}

// The editor can change a sample when it loads it. Make sure that each change is correct.
for ( const [ path, sample ] of Object.entries( samples ) ) {
	report.push( await check( path, sample ) );
}

// The test data must not change at all.
report.push( await check( 'Test data', testData ) );

document.querySelector( '#result' ).textContent = report.join( '\n\n' );

Open the test page with the Vite development server, for example at http://localhost:5173/test-data.html. The page shows the result for each sample and for the test data. The changes of a sample are the changes that the editor makes when it loads the sample. Make sure that each of them is correct, as step 4 of Making the test data describes. The page does not show the full output, so take the test data with the command of step 3. The test data must have no changes.

The local check differs from the check of Cloud Services in these points:

• It works only for a single-root editor. For a multi-root editor, create the editor from the roots of the test data. Then compare JSON.stringify( editor.getFullData( { trim: 'none' } ) ) with the test data.
• It does not create the second editor from the operations. In rare cases, a problem shows only in the upload.
• It removes the comment and suggestion markers first, because the test page has no data of comments and suggestions.
• The list of the plugins that Cloud Services removes must match your editor. The list depends on the version of Cloud Services, for example of an on-premises installation. The GET /editors/{bundle_version}/config request shows the list for an uploaded bundle.

The upload checks more than the test data. The most common other errors are:

• Editor should be accessible under the `CKEditorCS` name. – The bundle does not set the CKEditorCS global.
• Broken editor build. Details: … – The bundle stopped with an error, for example Unexpected token 'export' for an ES module or process is not defined.
• Broken editor build. Details: editor-source-element-already-used – The bundle has a multi-root editor, but the upload request does not set is_multi_root_editor to true. Depending on the configuration and the test data, the error can also be multi-root-editor-root-attributes-no-root or datacontroller-get-non-existent-root.
• plugincollection-plugin-not-found {"plugin":"…"} – The plugins or extraPlugins option of the configuration names a plugin that the bundle does not have.
• plugincollection-required {"plugin":"…","requiredBy":"…"} – A plugin of the bundle requires a plugin from removePlugins.

Refer to the Troubleshooting section of the Vite guide for the causes and the solutions.

A successful check means that the bundle gives the same content for the test data. The check does not test these things:

• The plugins in removePlugins, and the plugins that Cloud Services removes itself, for example the real-time collaboration plugins.
• Plugins that need network requests. Cloud Services blocks the requests during the check.
• Content that is not in the test data.
• Collaboration sessions, comments and suggestions, and revision history.
• The changes that the editor makes when it first loads a sample. The test data is the output of the editor, so it already contains these changes. The test page shows them.
• A plugin that changes the content only when the editor loads it, for example a post-fixer that changes straight quotes to typographic quotes. The test data already contains the result, so the check passes also without this plugin. Use the same editor class in your application and in the bundle, as the shared editor module of the Vite guide does.

Do these checks in addition:

• Run the upload in your CI pipeline, after the bundle build. The upload script of the Vite guide exits with an error when the check fails, so the pipeline stops.
• Run the local check in CI with a headless browser, for example with Playwright.
• After you deploy a new bundle, open a document in your application. Then use the GET /collaborations/{document_id}/details request: current_session.bundle_version must be the new bundle version.
• Export the document with the GET /collaborations/{document_id} request, and compare the result with the content in the browser.

The placeholder plugin of the Vite example project stores a placeholder as <span class="placeholder">{first name}</span>. Older documents of the application store the name without braces. One sample comes from such a document. The test page shows this sample and its output:

• The sample: <p>Hello <span class="placeholder">first name</span>!</p>
• The output: <p>Hello&nbsp;<span class="placeholder">{irst nam}</span>!</p>

An upload with the sample as the test data fails, and its data.convertedData field shows the same output. The output has two changes:

•   before the placeholder is an expected change. A space next to an inline widget becomes  .

• {irst nam} is data loss. The converter of the plugin always removes the first and the last character, to remove the braces:

  // Extract the name from "{name}".
  const name = viewElement.getChild( 0 ).data.slice( 1, -1 );

  Copy code

If you take the test data from this output, the upload does not find the bug. When the editor loads {irst nam}, the converter removes only the braces, and the output does not change. Thus the upload passes. Only the comparison with the sample shows the data loss.

The fix removes the braces only if they are there:

// src/editor/plugins/placeholder/placeholderediting.js

// ...

			model: ( viewElement, { writer } ) => {
				const text = viewElement.getChild( 0 ).data;

				// The name is in braces, for example "{first name}". Older documents have it without braces.
				const name = text.startsWith( '{' ) && text.endsWith( '}' ) ? text.slice( 1, -1 ) : text;

				return writer.createElement( 'placeholder', { name } );
			}

// ...

After the fix, the test page shows correct content in the current format for the sample: <p>Hello&nbsp;<span class="placeholder">{first name}</span>!</p>. The sample has only one paragraph, so this change is the full output. Add this output to the test data, and keep the old document as a sample. The upload with this test data passes.