People are remarkably capable of generating their own goals, beginning with child’s play and continuing into adulthood. Despite considerable empirical and computational work on goals and goal-oriented behaviour, models are still far from capturing the richness of everyday human goals. Here we bridge this gap by collecting a dataset of human-generated playful goals (in the form of scorable, single-player games), modelling them as reward-producing programs and generating novel human-like goals through program synthesis. Reward-producing programs capture the rich semantics of goals through symbolic operations that compose, add temporal constraints and allow program execution on behavioural traces to evaluate progress. To build a generative model of goals, we learn a fitness function over the infinite set of possible goal programs and sample novel goals with a quality-diversity algorithm. Human evaluators found that model-generated goals, when sampled from partitions of program space occupied by human examples, were indistinguishable from human-created games. We also discovered that our model’s internal fitness scores predict games that are evaluated as more fun to play and more human-like.
Automatically predicting ICD-10 codes from clinical notes using machine learning models can reduce the burden of manual coding. However, existing methods often overlook the semantic relationships between ICD-10 codes, resulting in inaccurate evaluations when clinically similar codes are considered completely different. Traditional evaluation metrics, which rely on equality-based matching, fail to capture the clinical relevance of predicted codes. This study introduces NNBSVR (Neural Network-Based Semantic Vector Representations), a novel approach for generating semantic-based vector representations of ICD-10 codes. Unlike traditional approaches that rely on exact code matching, NNBSVR incorporates contextual and hierarchical information to enhance both prediction accuracy and evaluation methods. We validate NNBSVR using intrinsic and extrinsic evaluation methods. Intrinsic evaluation assesses the vectors’ ability to reconstruct the ICD-10 hierarchy and identify clinically meaningful clusters. Extrinsic evaluation compares our relevancy-based approach, which includes customized evaluation metrics, to traditional equality-based metrics on an ICD-10 code prediction task using a 9.57 million clinical notes corpus. NNBSVR demonstrates significant improvements over equality-based metrics, achieving a 9.81% gain in micro-F1 score on the training set and a 12.73% gain on the test set. A manual review by medical experts on a sample of 10,000 predictions confirms an accuracy of 92.58%, further validating our approach. This study makes two significant contributions: first, the development of semantic-based vector representations that encapsulate ICD-10 code relationships and context; second, the customization of evaluation metrics to incorporate clinical relevance. By addressing the limitations of traditional equality-based evaluation metrics, NNBSVR enhances the automated assignment of ICD-10 codes in clinical settings, demonstrating superior performance over existing methods.
In this paper, a novel data-driven optimal control method based on reinforcement learning concepts is introduced. The proposed algorithm performs as a workaround to solving the Hamilton–Jacobi–Bellman equation. The main concept behind the proposed algorithm is the so-called IsoCost hypersurface (ICHS), which is a hypersurface in the state space of the system formed by points from which a specific amount of cost is spent by the control strategy in order to asymptotically stabilize the system. The fact that the control strategy requires to spend equal costs in order to stabilize all points on an ICHS is the reason for the naming of the IsoCost concept. Additional assumptions and definitions are mentioned before providing the theory of ICHS optimality. This theory proves, by contradiction, that the ICHS corresponding to the optimal control policy surrounds the ICHSs corresponding to other non-optimal control solutions. This paves the path to finding the optimal control solution using dynamic programming. The proposed method is implemented on the linear, fixed-base inverted pendulum, cart-pole and torsional pendulum bar system models and the results are compared with that of literature. The performance of this method in terms of cost, settling time and computation time is shown using numeric and illustrative comparisons.