Tutorial 3: Blueprint and C++ integration

To have AI write the page, see Tutorial 1; to write it by hand, see Tutorial 4.

Game state, rules and saved data belong to Unreal; the page renders state and submits actions. Check every parameter from the page again in Unreal.

Before you start: download the dependencies

Click the WebUIStudio button on the Level Editor toolbar and click Download in the OrionBrowser Dependencies window. Without the dependencies an interface shows only a line of text and the project cannot be packaged. Skip this if you have already done it; details are in Installation.

Choose a route

Use Blueprint only when the state is a few numbers, strings and switches. Use C++ when the state has arrays and nesting, or you need high-frequency data or guaranteed delivery. Use a C++ base class with a Blueprint subclass when programmers own the validation and designers configure sounds and assets in Blueprint.

The host can be a plain widget (an Orion WebUI in a Widget Blueprint), a CommonUI page (parent class Orion WebUI Activatable Widget), or an InstantScreen, where several pages share one browser. For a first integration use Blueprint only with a plain widget.

The contract used here

The settings panel of Tutorials 1 and 4. The AppId is SettingsPanel, and the production files are in the project's Content/UI/WebUI/SettingsPanel/dist.

A message the page sends with emit arrives in On Web Event. One sent with call arrives in On Web Request and must be answered.

Blueprint only: from scratch

1. Create the App Definition

1. Right-click in the Content Browser, choose Miscellaneous → Data Asset, pick the class OrionWebUIAppDefinition (search for Orion), and name it DA_SettingsPanel.
2. Open it and set:

Leave everything else at its default. The common properties are listed in Tutorial 4.

2. Create the Widget that hosts the page

1. Create a User Widget Blueprint named WBP_SettingsPanel.
2. In Designer, find Orion WebUI in the palette, drag it in, name it WebUI, enable Is Variable, and let it fill its parent.
3. In Details, set App Definition to DA_SettingsPanel. Designer shows the page.

3. Show it

In the Player Controller (or wherever you manage interfaces):

1. Create Widget (Class WBP_SettingsPanel, Owning Player Self) and store the result in a variable.
2. Add to Viewport.
3. Set Input Mode Game And UI: connect Self to Player Controller (the node does nothing while it is empty) and the WebUI of this Widget to In Widget to Focus, so that keyboard input reaches the page.
4. Set Show Mouse Cursor to true.

To close, do the reverse: Remove from Parent, Set Input Mode Game Only (connect its Player Controller too; inside the Widget Blueprint, use Get Owning Player), hide the cursor. Create an interface only once.

4. Receive actions: On Web Event

Select WebUI in Designer and click the plus next to On Web Event under Events at the bottom of Details. The event gives Event Name and Payload Json. Connect a Switch on Name, select it, and add one output per event name under Pin Names in Details. The node shows pin names with spaces (Settings Panel.set Volume); the comparison still uses the name you entered.

settingsPanel.ready: call PublishState from the next section.

settingsPanel.setVolume:

1. Get Json Number: connect Payload Json to Json, and enter volume as Field.
2. Branch on Valid. If it is false, do nothing.
3. Branch on the value being between 0 and 100.
4. If it passes: Round it to an integer, store it in the variable Volume, apply it to the game's volume setting, and call PublishState.
5. If it does not pass, call PublishState anyway to correct the page's display.

settingsPanel.setQuality: read quality with Get Json String, check Valid, accept only low, medium and high with a Switch on String, then store it and call PublishState.

settingsPanel.close: run the closing steps of section 3.

Connect no business logic to the Default output. The plugin's own notifications (for example __orion.inputReady) arrive here too; let them fall through to Default.

5. Publish state

Create a function PublishState, with the variables StateRevision (Integer64), Volume (Integer), Quality (String) and Culture (String):

1. Add one to StateRevision.
2. Set Json Number: enter {} as Json, stateRevision as Field, and connect StateRevision to Value.
3. Connect the return value to Json of the next Set Json Number, with Field volume.
4. In the same way add two Set Json String nodes for quality and culture.
5. Call Post Retained Latest Event to Web on WebUI: Event Name is ue:settingsPanel.state, and Payload Json is the return value of the last node.

Always send the full state; stateRevision only increases. A reloaded page sends settingsPanel.ready again, so always publish when you receive it.

6. Requests that need a result: On Web Request

Add the On Web Request event of WebUI. It gives Request and Response. Break Request (click the arrow at the bottom of the Break node to expand it) to get Name and Payload Json, then connect a Switch on Name:

settingsPanel.resetDefaults:

• When it is allowed: set the variables back to their defaults and apply them, call PublishState, then call Resolve Json on Response with {} as Data Json.
• When it is not: call Reject on Response, with your own error code as Error Code (for example E_LOCKED) and a reason for the player as Error Message.

Default output: call Reject on Response with the code E_UNKNOWN_REQUEST.

Rules:

• Each request is answered once. Call Resolve Json only after Unreal has really accepted the action.
• When the result takes a while (waiting for a save to finish, for example), call Defer first, store Response in a variable, and answer later. If no answer arrives within BridgeCallTimeoutSeconds of the App Definition (15 seconds by default), the page receives E_TIMEOUT.
• A request that gets neither an answer nor Defer succeeds with {} by default (Auto Resolve Unhandled Requests on WebUI), so always Reject explicitly on the Default output.

7. Ready and errors

• On Web Ready: the page and Unreal are connected; it fires again after the page reloads. You can publish the first state here.
• On Web Error: bridge errors, blocked navigation and the like. While developing, connect it to Print String; look here first when something is wrong.

8. Presentation state and cleanup

The page becomes visible by itself once it is ready. If a Blueprint-only interface calls Set Presentation Lifecycle State, use Visible alone, with 0 as Presentation Revision. To hide the interface for a while, change the Widget's visibility or remove it.

When the Widget is destroyed, clean up what you used in Event Destruct:

• Deactivate Control Sound Policy (with Self as Policy Owner)
• Clear Native Surfaces and Release Texture Resources
• Unbind Overlay Widget of WorldUI
• Your own timers

JSON in Blueprint

Get Json String / Number / Boolean reads one top-level field; Valid is false when the field is missing or has another type. Set Json String / Number / Boolean writes one field and returns the new JSON. They handle top-level fields only, and they escape quotes and line breaks correctly; do not concatenate text entered by the player with Append.

For arrays or nested objects: flatten them into top-level fields (such as slot0Name, slot0Count and slotCount); or enable the engine's Json Blueprint Utilities plugin, build the object, and publish it as a string; or move that part of the state into C++.

Common features in Blueprint

Control sounds

Hover and click sounds are played by Unreal. Every clickable element of the page has a control Id (data-orion-control-id), and a policy in Blueprint decides what to play.

1. In the Widget Blueprint that hosts the page, add a variable ControlSounds of type OrionWebUIControlSoundPolicy and compile.
2. Fill in its default value:

3. Event Construct: call Activate Control Sound Policy on WebUI with Self as Policy Owner, an empty Context Id, and ControlSounds as Policy.
4. Event Destruct: call Deactivate Control Sound Policy on WebUI with Self as Policy Owner.

Notes:

• To silence one control, add an override, enable the matching Override switch, and clear Enabled of that sound.
• To change the policy (when the volume changes, for example), call Activate Control Sound Policy again for the same Policy Owner.
• When an area of the page needs another policy, the page writes data-orion-sound-context="Shop" on the area's root element, and Blueprint activates a second policy with a different Policy Owner and Shop as Context Id.
• To check the Ids, select WebUI in Designer and set Design Time Control Id Preview Mode to All Controls.

Sounds with a business meaning

Sounds such as "saved" or "not enough credit" are requested by the page at the right moment; the sound assets are configured in Unreal:

1. Create an OrionWebUISoundManifest data asset and register StableId (for example settings.saved) and Sound in Entries.
2. Assign it to Sound Manifest of the App Definition.
3. The page calls api.playSound("settings.saved"). Blueprint can also call Play Sound by Id on WebUI.

Text and language

The simplest way is to publish culture in the state and let the page use its own bilingual dictionary, as the supplied sample does. To use Unreal's localization pipeline, create an OrionWebUITextCatalog data asset, register keys (for example settingsPanel.title) with Unreal text in Texts, and assign it to Text Catalog of the App Definition; the page looks text up by key.

Runtime images

To show a Texture2D (an avatar, an icon generated at run time) in the page:

1. Call Request Texture Resource on WebUI: a stable name as Stable Id (for example player.avatar), the texture as Texture, and Options built with a Make node, with Max Output Dimension set for the displayed size.
2. Bind On Texture Resource Completed of WebUI. When Status is Ready, write Url from Handle into the state and publish it; the page uses it for an image.
3. When it is no longer needed, call Release Texture Resource.

Requesting the same Stable Id again yields a new address, so the page uses only the latest one. When pixels change in place, raise Source Revision in Options. Virtual textures are not supported.

Live pictures: Native Surface

Draws a Render Target, a scene capture or a UI material directly at a position in the page:

1. The page puts a placeholder element at the position and binds it under a stable name (for example station.capture).
2. Blueprint calls Set Native Surface Texture on WebUI (Surface Id is the same name, Texture is the Render Target), or Set Native Surface Material. Position and size follow the placeholder.
3. When finished, call Clear Native Surface.

There are also Set Native Surface Tint (tint and opacity), Set Native Surface Composition Layer (Above Browser by default; Below Browser draws under the page and needs transparency enabled on the App Definition), and On Native Surface Layout Changed (pause the scene capture while Visible is false). The native picture receives no mouse input; put rounded corners and masks into the UI material.

WorldUI: labels and interaction that follow Actors

Each local player has one page that carries every world label; Actors carry a data component only.

1. Create an OrionWebUIWorldElementDefinition data asset and set what you need: ElementType (the page uses it to choose a renderer), MaxVisibleDistance (0 disables distance culling), bCheckOcclusion, bClampToViewport, bScaleWithDistance. For clicks, enable bInteractive and enter action names in AllowedActions (for example collect); MaxInteractionDistance is 300 by default.
2. Add the component to the Actor (search for World Element; the class is OrionWebUIWorldElementComponent): assign Definition and enter Element Key.
3. Provide text and state with Set Payload Json; the default renderer in the page reads label and actionLabel. Show or hide with Set Element Visible.
4. After the interface is created, get the local player subsystem OrionWebUIWorldSubsystem and call Bind Overlay Widget with WebUI.
5. Bind On Interaction of the component (it gives Player, Action Id and Payload Json), decide whether the game rules allow it, then change state. A networked game sends a validated server request as well.
6. Call Unbind Overlay Widget when the interface is destroyed.

Remote websites

Use a separate Orion Browser widget for external websites. It has no game bridge; do not mix it with Orion WebUI. Open an address with Initial URL or Load URL, navigate with Go Back / Go Forward / Reload / Stop Load, and show a failed view on On Load Error, as the sample's WBP_OrionWebsite does.

Host it with CommonUI, without C++

1. Choose Orion WebUI Activatable Widget as the parent class when you create the Widget Blueprint.
2. Add an Orion WebUI child. Its name must be WebUI.
3. In Class Defaults set App Definition, Control Sound Policy (no manual activation needed), Input Config (Menu by default), and optionally Input Manifest.
4. In the event graph, override HandleWebUIEvent, HandleWebUIRequest and HandleWebUIReady. The logic is the same as for On Web Event and On Web Request above.
5. Push it onto your CommonUI stack.

When an action of the Input Manifest fires, ue:commonAction is sent to the page and HandleCommonAction is called; when the page already submits an action for it, Blueprint must not run it again. Override HandleWebUIBackAction and return true to say you handled back; when nothing handled it, Deactivate on Back Action decides whether the page closes.

Blueprint node reference

All of these are on WebUI (Orion WebUI).

C++: from scratch

Module dependencies

The plugin's browser modules support Win64 only and take no part in Server builds. Put the code that hosts interfaces into a module built for clients only:

// MyGameUI.Build.cs
PublicDependencyModuleNames.AddRange(new string[] { "Core", "CoreUObject", "Engine", "UMG", "OrionWebUI", "OrionWebUIWidget" });
PrivateDependencyModuleNames.AddRange(new string[] { "Json" });
// For CommonUI pages add: "CommonUI", "OrionWebUICommonUI"
// For WorldUI add: "OrionWebUIWorld" (the component, available on Server) and "OrionWebUIWorldWidget" (the subsystem)

When the project has a Server target, give this module "TargetDenyList": ["Server"] and "PlatformAllowList": ["Win64"] in its descriptor in the .uproject or .uplugin.

Plain widget: UUserWidget with BindWidget

A Widget Blueprint uses this class as its parent and has an Orion WebUI named WebUI in Designer.

// SettingsPanelWidget.h
#pragma once

#include "Blueprint/UserWidget.h"
#include "OrionWebUIDefinitions.h"

#include "SettingsPanelWidget.generated.h"

class UOrionWebUIWidget;

UCLASS(Abstract, Blueprintable)
class MYGAMEUI_API USettingsPanelWidget : public UUserWidget
{
	GENERATED_BODY()

protected:
	virtual void NativeConstruct() override;
	virtual void NativeDestruct() override;

private:
	UFUNCTION()
	void HandleWebReady();

	UFUNCTION()
	void HandleWebEvent(FName EventName, const FString& PayloadJson);

	UFUNCTION()
	void HandleWebRequest(const FOrionWebUIBridgeRequest& Request, UOrionWebUIResponseHandle* Response);

	void PublishState();

protected:
	UPROPERTY(BlueprintReadOnly, meta=(BindWidget))
	TObjectPtr<UOrionWebUIWidget> WebUI;

	UPROPERTY(EditDefaultsOnly, Category="Settings Panel")
	FOrionWebUIControlSoundPolicy ControlSounds;

private:
	int64 StateRevision = 0;
	int32 Volume = 80;
	FString Quality = TEXT("high");
	FString Culture = TEXT("en");
};

// SettingsPanelWidget.cpp
#include "SettingsPanelWidget.h"

#include "Dom/JsonObject.h"
#include "OrionWebUIWidget.h"
#include "Policies/CondensedJsonPrintPolicy.h"
#include "Serialization/JsonReader.h"
#include "Serialization/JsonSerializer.h"
#include "Serialization/JsonWriter.h"

namespace SettingsPanel
{
	static const FName StateEvent(TEXT("ue:settingsPanel.state"));
	static const FName ReadyEvent(TEXT("settingsPanel.ready"));
	static const FName SetVolumeEvent(TEXT("settingsPanel.setVolume"));
	static const FName SetQualityEvent(TEXT("settingsPanel.setQuality"));
	static const FName CloseEvent(TEXT("settingsPanel.close"));
	static const FName ResetDefaultsRequest(TEXT("settingsPanel.resetDefaults"));
}

void USettingsPanelWidget::NativeConstruct()
{
	Super::NativeConstruct();

	if (!WebUI)
	{
		return;
	}
	WebUI->OnWebReady.AddUniqueDynamic(this, &ThisClass::HandleWebReady);
	WebUI->OnWebEvent.AddUniqueDynamic(this, &ThisClass::HandleWebEvent);
	WebUI->OnWebRequest.AddUniqueDynamic(this, &ThisClass::HandleWebRequest);
	WebUI->ActivateControlSoundPolicy(this, NAME_None, ControlSounds);
}

void USettingsPanelWidget::NativeDestruct()
{
	if (WebUI)
	{
		WebUI->DeactivateControlSoundPolicy(this);
		WebUI->OnWebReady.RemoveDynamic(this, &ThisClass::HandleWebReady);
		WebUI->OnWebEvent.RemoveDynamic(this, &ThisClass::HandleWebEvent);
		WebUI->OnWebRequest.RemoveDynamic(this, &ThisClass::HandleWebRequest);
	}

	Super::NativeDestruct();
}

void USettingsPanelWidget::HandleWebReady()
{
	PublishState();
}

void USettingsPanelWidget::HandleWebEvent(FName EventName, const FString& PayloadJson)
{
	if (EventName == SettingsPanel::ReadyEvent)
	{
		PublishState();
		return;
	}
	if (EventName == SettingsPanel::CloseEvent)
	{
		RemoveFromParent();
		return;
	}

	TSharedPtr<FJsonObject> Payload;
	if (!FJsonSerializer::Deserialize(TJsonReaderFactory<>::Create(PayloadJson), Payload) || !Payload.IsValid())
	{
		return;
	}

	if (EventName == SettingsPanel::SetVolumeEvent)
	{
		double RequestedVolume = 0.0;
		if (Payload->TryGetNumberField(TEXT("volume"), RequestedVolume) && RequestedVolume >= 0.0 && RequestedVolume <= 100.0)
		{
			Volume = FMath::RoundToInt32(RequestedVolume);
			// Apply the volume to the game here.
		}
		PublishState();
		return;
	}
	if (EventName == SettingsPanel::SetQualityEvent)
	{
		FString RequestedQuality;
		if (Payload->TryGetStringField(TEXT("quality"), RequestedQuality)
			&& (RequestedQuality == TEXT("low") || RequestedQuality == TEXT("medium") || RequestedQuality == TEXT("high")))
		{
			Quality = RequestedQuality;
		}
		PublishState();
	}
}

void USettingsPanelWidget::HandleWebRequest(const FOrionWebUIBridgeRequest& Request, UOrionWebUIResponseHandle* Response)
{
	if (!Response)
	{
		return;
	}
	if (Request.Name == SettingsPanel::ResetDefaultsRequest)
	{
		Volume = 80;
		Quality = TEXT("high");
		PublishState();
		Response->ResolveJson(TEXT("{}"));
		return;
	}
	Response->Reject(TEXT("E_UNKNOWN_REQUEST"), TEXT("Unknown settings panel request."));
}

void USettingsPanelWidget::PublishState()
{
	if (!WebUI)
	{
		return;
	}

	const TSharedRef<FJsonObject> State = MakeShared<FJsonObject>();
	State->SetNumberField(TEXT("stateRevision"), static_cast<double>(++StateRevision));
	State->SetNumberField(TEXT("volume"), Volume);
	State->SetStringField(TEXT("quality"), Quality);
	State->SetStringField(TEXT("culture"), Culture);

	FString Json;
	FJsonSerializer::Serialize(State, TJsonWriterFactory<TCHAR, TCondensedJsonPrintPolicy<TCHAR>>::Create(&Json));
	WebUI->PostRetainedLatestEventToWeb(SettingsPanel::StateEvent, Json);
}

The answering rules are the same as in Blueprint; to answer asynchronously, call Response->Defer() first and keep the pointer. FName comparison ignores case and the page does not, so spell an event name one way throughout the project. If you do not want to parse JSON yourself, bind the native delegate OnWebEventParsed with AddUObject; it gives a TSharedPtr<FJsonObject>.

CommonUI pages

When the page enters a CommonUI stack, derive from UOrionWebUIActivatableWidget; the base class handles loading the app, the sound policy, input actions and back. The Widget Blueprint contains an Orion WebUI child named WebUI; configure AppDefinition, InputManifest, ControlSoundPolicy and InputConfig in Class Defaults, and push the page only through your CommonUI stack, never with AddToViewport.

// SettingsPanelScreen.h
#pragma once

#include "OrionWebUIActivatableWidget.h"

#include "SettingsPanelScreen.generated.h"

UCLASS(Abstract, Blueprintable)
class MYGAMEUI_API USettingsPanelScreen : public UOrionWebUIActivatableWidget
{
	GENERATED_BODY()

protected:
	virtual void NativeOnActivated() override;
	virtual void HandleWebUIReady_Implementation() override;
	virtual void HandleWebUIEvent_Implementation(FName EventName, const FString& PayloadJson) override;
	virtual void HandleWebUIRequest_Implementation(const FOrionWebUIBridgeRequest& Request, UOrionWebUIResponseHandle* Response) override;
	virtual bool HandleWebUIBackAction_Implementation() override;

private:
	void PublishState(bool bForceRepublish);

private:
	FString PublishedStateJson;
	int64 StateRevision = 0;
};

// SettingsPanelScreen.cpp (excerpt)
void USettingsPanelScreen::NativeOnActivated()
{
	Super::NativeOnActivated();
	PublishState(true);			// State may have changed while deactivated: publish from current data
}

void USettingsPanelScreen::HandleWebUIReady_Implementation()
{
	Super::HandleWebUIReady_Implementation();
	PublishState(true);
}

void USettingsPanelScreen::HandleWebUIEvent_Implementation(FName EventName, const FString& PayloadJson)
{
	if (EventName == StandardPresentationReadyEventName || EventName == StandardPresentationAppliedEventName)
	{
		return;
	}
	// The rest is the same as HandleWebEvent of the plain widget; get the widget with GetWebUIWidget()
}

bool USettingsPanelScreen::HandleWebUIBackAction_Implementation()
{
	return false;	// Return true when you handled back yourself; the default close does not run
}

Four channels for publishing state

Allocate revisions with UOrionWebUIWidget::AllocatePresentationRevision(); they are a separate number space from the stateRevision of business state. Do not allocate a new revision when the content did not change. An acknowledgement from the page only says that a revision arrived or is shown; it does not mean business success.

Resource interfaces

In C++, use RequestTextureResource(StableId, Texture, Options) (OnTextureResourceCompleted fires on completion), SetNativeSurfaceTexture and SetNativeSurfaceMaterial. For CPU pixels you already have, use RequestPixelResource(StableId, FOrionWebUIPixelSource, Options), which is C++ only; for a material that samples transient resources, use SetNativeSurfaceMaterialWithDependencies(SurfaceId, Material, Dependencies). When you change the Render Target, create a new dynamic material instance and call SetNativeSurfaceMaterial; do not modify an instance that is still bound.

WorldUI

#include "OrionWebUIWorldSubsystem.h"

if (ULocalPlayer* LocalPlayer = GetOwningLocalPlayer())
{
	if (UOrionWebUIWorldSubsystem* WorldUI = LocalPlayer->GetSubsystem<UOrionWebUIWorldSubsystem>())
	{
		WorldUI->BindOverlayWidget(WebUI);
	}
}

OnInteraction of the component is a dynamic multicast delegate (APlayerController* Player, FName ActionId, const FString& PayloadJson). For a label that belongs to one local player, use SetOwningLocalPlayer of the component.

A C++ base class with a Blueprint subclass

Write the validation that must not be bypassed in C++, and leave what designers tune to Blueprint:

UCLASS(Abstract, Blueprintable)
class MYGAMEUI_API USettingsPanelWidget : public UUserWidget
{
	GENERATED_BODY()

protected:
	UFUNCTION(BlueprintNativeEvent, Category="Settings Panel")
	void OnVolumeAccepted(int32 NewVolume);

	UPROPERTY(EditDefaultsOnly, BlueprintReadOnly, Category="Settings Panel")
	bool bResetEnabled = true;
};

• Call the BlueprintNativeEvent only after validation passes; a Blueprint override cannot skip it.
• Give each fixed button one event and one enable switch, and publish the switches to the page with the full state.
• The sound policy, App Definition and Input Manifest are EditDefaultsOnly properties configured in Class Defaults of the Blueprint subclass.
• Do not rename, change the signature of, or delete events that Blueprints already override.

InstantScreen overview

InstantScreen lets several interfaces on one layer share one resident browser, and shows an interface only once its state, resources and first frame are ready. For a single panel, a plain widget is enough. It needs instant-screen.config.ts in the page, assets such as OrionInstantScreenDefinition, an Orion InstantScreen Runtime Host in the root layout, and interfaces whose parent derives from UOrionInstantScreenActivatableWidget; see the InstantScreen reference shipped with the plugin.

Troubleshooting