{"entities":{"Q7368653":{"pageid":31557827,"ns":120,"title":"Item:Q7368653","lastrevid":108149117,"modified":"2026-11-02T01:04:27Z","type":"item","id":"Q7368653","labels":{"en":{"language":"en","value":"Gtheory4LLM"}},"descriptions":{"en":{"language":"en","value":"Generalizability Theory for LLM Subjective Tasks"}},"aliases":{},"claims":{"P31":[{"mainsnak":{"snaktype":"value","property":"P31","hash":"438d0eef0223375d3a92def8ffc6a10a8bf1ca95","datavalue":{"value":{"entity-type":"item","numeric-id":57080,"id":"Q57080"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","id":"Q7368653$F7EDE235-D2CC-46BC-AB0D-12AADFA81DE5","rank":"normal"}],"P1460":[{"mainsnak":{"snaktype":"value","property":"P1460","hash":"908c3454b3659c4b140ccce33c5aee31081edc8d","datavalue":{"value":{"entity-type":"item","numeric-id":5976450,"id":"Q5976450"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","id":"Q7368653$6095E6E8-E5C8-4FAA-9C73-24330A5B0DCD","rank":"normal"}],"P114":[{"mainsnak":{"snaktype":"value","property":"P114","hash":"46bebcbff353210c0cda7715f07aada2d0e07183","datavalue":{"value":{"entity-type":"item","numeric-id":56774,"id":"Q56774"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","id":"Q7368653$D9F09C45-A1F3-4013-9431-8444DE24EF20","rank":"normal"}],"P132":[{"mainsnak":{"snaktype":"value","property":"P132","hash":"189c587a08f42dbc3cc769bb9feba00c37905fbc","datavalue":{"value":"0.2.0","type":"string"},"datatype":"string"},"type":"statement","qualifiers":{"P28":[{"snaktype":"value","property":"P28","hash":"c318a2b1abe33e6ee4074218389bdfba27d6b7bf","datavalue":{"value":{"time":"+2026-10-04T00:00:00Z","timezone":0,"before":0,"after":0,"precision":11,"calendarmodel":"http://www.wikidata.org/entity/Q1985727"},"type":"time"},"datatype":"time"}]},"qualifiers-order":["P28"],"id":"Q7368653$48CDFC22-C3F0-4AB5-9109-772D4E1DE726","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P132","hash":"b542975f358e64266fc8bf3287bc018a1e05a763","datavalue":{"value":"PACKAGES.rds","type":"string"},"datatype":"string"},"type":"statement","qualifiers":{"P28":[{"snaktype":"value","property":"P28","hash":"3520d94d7d9f2048b6222ac16b57c86777514254","datavalue":{"value":{"time":"+2026-10-11T00:00:00Z","timezone":0,"before":0,"after":0,"precision":11,"calendarmodel":"http://www.wikidata.org/entity/Q1985727"},"type":"time"},"datatype":"time"}]},"qualifiers-order":["P28"],"id":"Q7368653$5321C7F0-CCE6-43D9-9951-54E161B9DCA8","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P132","hash":"31398def8e59a76a9969090f642ae3e3d8fd8119","datavalue":{"value":"0.5.0","type":"string"},"datatype":"string"},"type":"statement","qualifiers":{"P28":[{"snaktype":"value","property":"P28","hash":"3520d94d7d9f2048b6222ac16b57c86777514254","datavalue":{"value":{"time":"+2026-10-11T00:00:00Z","timezone":0,"before":0,"after":0,"precision":11,"calendarmodel":"http://www.wikidata.org/entity/Q1985727"},"type":"time"},"datatype":"time"}]},"qualifiers-order":["P28"],"id":"Q7368653$F240AEF4-B358-4201-A04F-957B10AAAB83","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P132","hash":"b542975f358e64266fc8bf3287bc018a1e05a763","datavalue":{"value":"PACKAGES.rds","type":"string"},"datatype":"string"},"type":"statement","qualifiers":{"P28":[{"snaktype":"value","property":"P28","hash":"36fdc2afbe9017ab5ce05dc71c59b7f81bfa03a4","datavalue":{"value":{"time":"+2026-11-02T00:00:00Z","timezone":0,"before":0,"after":0,"precision":11,"calendarmodel":"http://www.wikidata.org/entity/Q1985727"},"type":"time"},"datatype":"time"}]},"qualifiers-order":["P28"],"id":"Q7368653$FECD7FD0-AEC7-4432-9CBE-E5E792F9F00A","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P132","hash":"2cd9663599cb012c24c42164f1aeb49c4d6af9f9","datavalue":{"value":"1.0.0","type":"string"},"datatype":"string"},"type":"statement","qualifiers":{"P28":[{"snaktype":"value","property":"P28","hash":"d3defaead3d95f9d6c8b30660db2d4a2f7c845b7","datavalue":{"value":{"time":"+2026-11-01T00:00:00Z","timezone":0,"before":0,"after":0,"precision":11,"calendarmodel":"http://www.wikidata.org/entity/Q1985727"},"type":"time"},"datatype":"time"}]},"qualifiers-order":["P28"],"id":"Q7368653$D8764CD7-C1B9-4E3F-B029-1DB245212010","rank":"normal"}],"P16":[{"mainsnak":{"snaktype":"value","property":"P16","hash":"48c9cf69d7fcf98abe9dda85651b347fad898d68","datavalue":{"value":{"entity-type":"item","numeric-id":159983,"id":"Q159983"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","id":"Q7368653$73A79270-732B-4D51-924B-21F18AA5A306","rank":"normal"}],"P19":[{"mainsnak":{"snaktype":"value","property":"P19","hash":"b0cd5559efe3012e6a4166d9fe48ca14b9c18270","datavalue":{"value":{"entity-type":"item","numeric-id":159983,"id":"Q159983"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","id":"Q7368653$A60423AE-C133-4AA8-9173-6549D6CAE636","rank":"normal"}],"P163":[{"mainsnak":{"snaktype":"value","property":"P163","hash":"8576d3d16efe97b9b62facebe80c3ed9e3b6aae2","datavalue":{"value":{"entity-type":"item","numeric-id":56621,"id":"Q56621"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","id":"Q7368653$3D36A437-89CE-4DE3-B04B-57550E0282A0","rank":"normal"}],"P342":[{"mainsnak":{"snaktype":"value","property":"P342","hash":"a0beaece152e3d4cfe5f01b8a4c02caff7e1e6cc","datavalue":{"value":{"entity-type":"item","numeric-id":56774,"id":"Q56774"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","qualifiers":{"P132":[{"snaktype":"value","property":"P132","hash":"def48c908801413f80687d8abfd49e1fdc4712b5","datavalue":{"value":"\u2265 4.5.0","type":"string"},"datatype":"string"}]},"qualifiers-order":["P132"],"id":"Q7368653$25A28DCB-D5E7-4FB0-A415-A48B1EECCFEC","rank":"normal"}],"P585":[{"mainsnak":{"snaktype":"value","property":"P585","hash":"2e3d92c2f016e6ac30256a8be95a03428ea2904b","datavalue":{"value":{"entity-type":"item","numeric-id":16686,"id":"Q16686"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","qualifiers":{"P132":[{"snaktype":"value","property":"P132","hash":"a6f5b075a1aafb70d8b444605c5b389f333c08c3","datavalue":{"value":"\u2265 1.6.0","type":"string"},"datatype":"string"}]},"qualifiers-order":["P132"],"id":"Q7368653$88E1835A-B224-47EB-8A7C-4B97BBC6F2A8","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P585","hash":"bf29a5ce9a21b014d0f45aa525e11abc07df43f9","datavalue":{"value":{"entity-type":"item","numeric-id":64892,"id":"Q64892"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","qualifiers":{"P132":[{"snaktype":"value","property":"P132","hash":"eb4bd6e6a4d0626a6273329c376a2c9e4d6d86f5","datavalue":{"value":"\u2265 2.22.11","type":"string"},"datatype":"string"}]},"qualifiers-order":["P132"],"id":"Q7368653$52341E4B-16AA-4D9A-B849-95BC49CB1CE9","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P585","hash":"763ab1af0d78af1e43710d686a298c3714e83ab4","datavalue":{"value":{"entity-type":"item","numeric-id":57371,"id":"Q57371"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","id":"Q7368653$0345B812-7146-4F6E-B5BE-81AD577E78E4","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P585","hash":"bd9baf02a50a5963b258028913207f83e0a26758","datavalue":{"value":{"entity-type":"item","numeric-id":57177,"id":"Q57177"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","id":"Q7368653$033ED94D-A22F-4668-95A9-DB4DD8D49349","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P585","hash":"24c3e3d95d6aceb19323da214a060d1bb0535ab2","datavalue":{"value":{"entity-type":"item","numeric-id":57178,"id":"Q57178"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","id":"Q7368653$9EEF1E38-590D-4F1B-B390-5F5F11EC8544","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P585","hash":"7aac00570a74583c448c8c2a1fe16ed9aaeff406","datavalue":{"value":{"entity-type":"item","numeric-id":57179,"id":"Q57179"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","id":"Q7368653$28A96A26-384F-4E8D-86C5-DBA027FFE44A","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P585","hash":"181574dfe61d92a30914c897a5022337ec55662b","datavalue":{"value":{"entity-type":"item","numeric-id":57412,"id":"Q57412"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","id":"Q7368653$C6C2C481-C22B-403A-A590-D4170C608DA2","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P585","hash":"ce7128dff4ab6b862855052e63bf609ca187bbcd","datavalue":{"value":{"entity-type":"item","numeric-id":57191,"id":"Q57191"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","id":"Q7368653$ED9797C4-0AB9-4282-A801-F575ED3A20AA","rank":"normal"}],"P223":[{"mainsnak":{"snaktype":"value","property":"P223","hash":"75e948bb647c525e0d65fbf6869dede7faaf04e4","datavalue":{"value":{"entity-type":"item","numeric-id":7368652,"id":"Q7368652"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","id":"Q7368653$1FFD010D-D795-494D-B74E-32336427E757","rank":"normal"}],"P229":[{"mainsnak":{"snaktype":"value","property":"P229","hash":"ae225ce44a5312ab40fdc1cabf33f76b9c8619a7","datavalue":{"value":"Gtheory4LLM","type":"string"},"datatype":"external-id"},"type":"statement","id":"Q7368653$1B64EE5F-D7E4-4E12-9D37-136D4D4EDEC8","rank":"normal"}],"P1459":[{"mainsnak":{"snaktype":"value","property":"P1459","hash":"da600cccde265dfdca599e2dddbec0460df69eb6","datavalue":{"value":"Studies the reliability and generalizability of subjective    judgments produced by large language models (LLMs), including annotation,    rating, and structured LLM-as-a-judge tasks. Specifies evaluator, prompt,    generation, and repeated-run facets through crossed or explicitly nested    random sources with configurable item interactions. Fits univariate models,    joint Gaussian models, and joint discrete models for binary, ordinal, and    unordered categorical outcomes, with source-specific covariance. Gaussian    models use exact balanced likelihood, with a bounded observed-row route    for incomplete univariate panels. Discrete models use dense or sparse    first-order Laplace approximation with Gaussian latent random effects.    Supported balanced decision studies compare evaluator, prompt, and    replication allocations using observed Gaussian or explicitly requested    latent binary and ordinal reliability, with random or fixed facets after    Brennan (2001). 'OpenMx' Gaussian fits report asymptotic Wald standard errors    for variance components and delta-method coefficient intervals when    curvature checks permit them; the 'lme4' route returns point estimates only.    Discrete fits offer optional full-information latent coefficient intervals,    subject to numerical and boundary checks. An object bootstrap refits    resampled objects under each fit's acceptance rule and gives percentile and    bias-corrected and accelerated intervals for components, coefficients and    decision studies. Provides fixed-layout call    effects, categorical score and category-vector reliability projections,    repeated-panel decision agreement, bounded cost-aware comparisons, and    Gaussian or latent binary/ordinal pilot precision planning, and categorical    point-estimator pilot operating characteristics. Joint    Gaussian-discrete fitting is not implemented. Includes three publicly    archived LLM annotation datasets covering hate-speech, mental-health,    and drug-review tasks. Discrete fitting retains conservative size limits;    the preflight report describes supported designs and computational limits.    Generalizability coefficients follow the variance-decomposition framework    of Brennan (2001) <doi:10.1007/978-1-4757-3456-0>.","type":"string"},"datatype":"string"},"type":"statement","id":"Q7368653$E695EC89-E86A-4EC7-8EB2-1CE7A738DDF8","rank":"normal"}],"P170":[{"mainsnak":{"snaktype":"value","property":"P170","hash":"d993fb7a21021da2bf866be4f280654436ffb61e","datavalue":{"value":{"time":"+2026-11-01T00:00:00Z","timezone":0,"before":0,"after":0,"precision":11,"calendarmodel":"http://www.wikidata.org/entity/Q1985727"},"type":"time"},"datatype":"time"},"type":"statement","id":"Q7368653$4D920650-8E8B-4E5E-954A-2EC4C5688176","rank":"normal"}]},"sitelinks":{"mardi":{"site":"mardi","title":"Gtheory4LLM","badges":[],"url":"https://portal.mardi4nfdi.de/wiki/Gtheory4LLM"}}}}}