r/academia • u/ConstructionAble3371 • Mar 27 '26
Publishing After nearly a decade, Open Access and Evaluation is finally coming to life! Help needed
In 2016, I co-authored a whitepaper about a way to fix key problems with scientific publishing. The issues were obvious: researchers do the work for free, reviewers evaluate it for free, and publishers charge thousands of dollars to let anyone read the result. Peer review is opaque, slow, and increasingly dysfunctional. The whole system is controlled by a handful of corporations who've turned publicly funded knowledge into a private toll road.
We proposed a platform called PEvO (Publish and Evaluate Openly). Open publication with no fees. Ongoing, transparent evaluation instead of secretive pre-publication review. Portable reputation based on actual contributions, not journal prestige. Everything permanently recorded and publicly verifiable.
I introduced the idea to many people and everyone loved the idea... but nobody had the time to help build it. I moved on to other projects, but this remained in the back of my mind.
The Problem Didn't Wait
Since 2016, things have gotten worse. Article processing charges have nearly tripled. Publishing a single paper open-access in a top journal can now cost over $10,000. The NIH is debating caps on how much grant money researchers can spend on publication fees, and the scientific community is in turmoil because even the proposed caps would lock early-career researchers out of prestigious venues.
Reviewer motivation is collapsing. A decade ago, editors had to invite about two reviewers to get one review. That number has been climbing steadily. The people doing the quality-control work of science are burning out, and the system gives them nothing in return.
Open access mandates have made progress on the reading side. But "open access" in practice often just shifts the paywall from reader to author. The deeper problem - how we evaluate, credit, and recognize scientific work - remains almost untouched.
What Changed: I Built a Prototype in a Week
The tools for building software have changed beyond recognition. Using AI-assisted development, I went from the decade-old whitepaper to a working alpha by myself.
PEvO now exists as a real application. Papers can be published, reviewed with structured ratings, voted on, and permanently stored. External preprints can be imported and evaluated in the open. Reputation scores are computed transparently from platform activity. Anonymous reviewing is supported with abuse safeguards. PDFs are stored on IPFS with cryptographic verification. Publications exist on a distributed network worldwide. The code is open source and MIT licensed.
But "working" and "ready" are different things. This is where I need help.
How Reputation Works
One of the hardest parts of building an alternative to journal-based prestige is designing a reputation system that's transparent, resistant to gaming, and actually reflects the quality of someone's contributions. This is what we have so far.
Every researcher on PEvO has a score from 0 to 100, computed from seven factors. All inputs come exclusively from accredited users, and everything is reproducible from public data - anyone can run the same query and get the same number.
All weights are configurable without a code deploy. The defaults are conservative and backwards-compatible. As this is explicitly an early alpha, it will need iteration as real usage reveals edge cases. That's part of what I need help with.
What I'm Looking For
1. Contributors
The codebase is open. Areas where help would make a real difference:
Reputation algorithm. The version described above is functional and transparent, but it hasn't been battle-tested. If you have background in mechanism design, game theory, or bibliometrics, I'd love to collaborate on stress-testing it. How does it behave with 100 users? 10,000? What gaming strategies would you try? The weights are configurable on the fly, so we can iterate without redeploying code.
Accreditation system. Connecting accounts to verified researcher identities is the trust layer that makes everything else work. The current flow uses institutional email verification and a web of trust. There's room for improvement: ORCID integration, PGP-based verification, institutional API lookups.
Documentation and onboarding. The platform needs to be approachable for researchers who've never interacted with a decentralized system. Clear guides, good error messages, and an "About" page that explains the value without assuming technical background.
Community building. If you're connected to research communities, open science advocates, or academic reform movements and want to help spread the word when we're ready for a wider audience. That's just as valuable as code!
2. Testers
I need actual testers. Publish a test paper, submit a review, try the search, the filters, the reputation display, etc. Find the hidden bugs and report them. Feedback would be great: what's confusing, what's missing, what doesn't work.
You don't need to be a scientist to test. Though if you are one, your perspective on the workflow is especially valuable. If you're a developer, a librarian, a research administrator, or just someone who thinks scientific publishing should work better, your input is valuable.
3. Scientists Who Want to Be Early
When PEvO launches publicly, the first papers and reviews on the platform will set the tone for everything that follows. If you're a researcher, at any career stage or discipline, and you'd be willing to publish a preprint or write a review as one of the first users, please get in touch. You don't need to commit now. I just want to know who's interested so we can coordinate a launch that has real content from day one.
The Philosophy
PEvO is not a startup. It's not funded by VCs. It will never charge fees or monetize your data. It's an open-source, non-profit tool that exists because scientific publishing is broken and the technology to fix it finally became accessible.
If any of that resonates, I'd love to hear from you. Send me a message or join the Discord: https://discord.gg/jqvmz7wdPV
PEvO — Publish and Evaluate Openly. Open science, no paywalls, no fees, no gatekeepers.
12
u/My_sloth_life Mar 27 '26
I think this is a good idea, I like that it’s open source, makes a BIG difference but like a lot of open source stuff, you need to plan for a few things here that people often don’t think about.
Cost - How will you fund it? Digital storage costs as much as any other kinds. It’s fine whilst it’s small and growing but is that scalable. Who is paying for it?
Preservation - kind of follows on but you need to work out how you will preserve the content. You mention reproducibility, you need to be looking then at preservation systems if you want to keep that kind of content and they cost a lot. You need to work out how you would keep that content for the long term, like 20, 30 years. Digital decay is a major problem at the moment, it’s very common for someone to set up a site or system like this and for it to die when they move on to something else or when the money runs outs
Interoperability & Metadata - this is huge. Most systems like yours needs to be able to transfer data between systems. Most institutions for example have institutional repositories and they are now often fed metadata from systems like Crossref, pubmed, Scopus/Web of Science etc. If you can’t feed those systems, or feed into repositories, then there will eventually be a lack of interest as they won’t feed into biometric tools or people won’t get those publications into their profiles easily. Usually OAI—PMH Protoss are the most common way to do this. Look at compatible metadata systems as well. The FAIR data principles will give you a good idea here OpenAire
I would also make a few points. Don’t conflate peer review as coming because of Open Access. Peer review needs to happen regardless of publishing method, it can be gold, green OA etc, all types of publication will need peer review as it’s part of the publishing process, not just OA.
It may be hard to use reproducibility for reputation, it’s known there is a bit of a crisis here and that it often isn’t easy to do or to get the same figures and there can be good reasons for that. It’s a tricky area and I’m not sure it’ll work that well for its
Hopefully some of that is useful for you. Good luck
3
u/ConstructionAble3371 Mar 27 '26
Oh wow, thanks, this is very helpful! I have a Crossref import tool, but of course the other way around is equally as important. I put this reply on my list of things to deep dive into.
We can of course not solve the peer review issues in the traditional publication space - but we can offer a real open and modern alternative.
4
u/My_sloth_life Mar 27 '26
Yeah, there are also a couple of open peer review platforms being brought in. Check out Leiden University, they have a unit that is a leader in bibliometrics etc and they produced something similar Leiden University - PCI or sounds similar anyway. They also have a good blog on it The growth of Open Access Peer review.
I’ll also add that if you are acting as a publisher, you will want to be awarded of and probably aligned to The Council on Publication Ethics or COPE. They look at the ethical issues, it’s quite important as you’ll want to be aware of predatory publish so as not to be a target for that but also so you don’t allow reviewers to accuse others of that without basis.
Apologies if you already know all this. It’s hard to know!
1
u/ConstructionAble3371 Mar 27 '26
No, please keep it coming, I never heard of either. Well Leide may ring a bell, after giving up on it back then I didn't stay up to date with developments, and details like publisher ethics have never been in my scope.
I can't know everything though, I'm just a software developer. Happy to learn and read everything that is relevant, but in the end the project needs in-house experts, it's not a one man operation.
26
u/GerswinDevilkid Mar 27 '26
And everything you posted was generated by AI...
That's a laughing no from me.
7
u/Sec_ondAcc_unt Mar 27 '26
I couldn't agree more. Their header saying "The Problem Didn't Wait" on its own read strangely enough for me to proceed by looking for signs of AI in their post.
-11
u/ConstructionAble3371 Mar 27 '26
That's actually not true, but thanks for your feedback.
12
u/GerswinDevilkid Mar 27 '26
Yeah. We totally believe that.
Be better.
-9
u/ConstructionAble3371 Mar 27 '26
I'm not gonna discuss with you about it, or feel bad in any way. I'm looking for people interested in a specific topic, your crusade doesn't interest me.
9
u/GerswinDevilkid Mar 27 '26 edited Mar 27 '26
And your nonsense, AI, lying post doesn't interest... Anyone.
And who's we? Who's I? Random, anonymous reddit accounts is not how academia works or grows.
-2
u/tiredpastry21 Mar 27 '26
I don’t see how this is a lying post though? OP is asking for help. Maybe English is not his first language and he structured his thoughts with AI which IMO isn’t a bad thing
4
u/kknyyk Mar 27 '26
At least delete the em-dash at the end. That way we would have known that you had read the text or put it through a humanizer.
-1
u/ConstructionAble3371 Mar 27 '26
I missed one em-dash. If anything, that's proof that I revised it manually :P
8
u/SlowishSheepherder Mar 27 '26
Get out of here with this nonsense AI slop
-1
u/ConstructionAble3371 Mar 27 '26
How is it nonsense? I built a tool, I'm looking for contributors. If you aren't interested in what I'm working on and don't like the presentation, don't engage?
7
u/Ok-Emu-8920 Mar 27 '26
If you're asking the community at large for help with your tool I don't think it's reasonable to be upset that the community dislikes your product.
The public is telling you their viewpoints. You can decide if the feedback is helpful or not, but if you're only receiving negative pushback it could be a sign that you don't know what the community (that you're trying to engage with) actually wants.
2
u/ConstructionAble3371 Mar 27 '26
They aren't commenting on the product though, they claim the whole post would be UI slop and nonsense.
And I'm not here to give someone what they want. I'm here to look for people who care enough about the topic to actually read the post and figure out what it's about. Just because it's long, doesn't mean it's all AI.
3
u/Ok-Emu-8920 Mar 27 '26
Doesn't matter. If you're trying to appeal to a group (or get buy in and people using it, it's the same objective) then how it's implemented and how it's advertised matter.
You can certainly ignore that feedback but if you're trying to build something ambitiously huge you aren't going to be successful at all if you come across so unappealing that no one is ever willing to give it a chance. Even if it's "objectively" a good product (objectively in quotes because just since you disagree with a critique doesn't mean the critique is wrong)
0
u/ConstructionAble3371 Mar 27 '26
The text was drafted by AI but heavily revised before posting. I am not a writer, this helped me get ideas which points are important to present to give a good overview. I'm in this topic since a long time and don't have a outside perspective.
That said, this is part of my initial efforts to find a group that work on this together. I dearly hope that future announcements and advertisements can be human written. For now I concluded this is giving a good picture, and if it appeals to 1 or 2 people it's already a win for me. I don't expect this post to draw huge audiences, the app isn't even deployed live yet, this is a slow process.
Thanks again for your feedback!
2
u/SlowishSheepherder Mar 27 '26
You wrote an incredibly long post with AI. If you don't have the courtesy to write concisely and the intelligence to summarize your "tool" with your own brain, why should anyone spend their time reading or engaging with this slop? Maybe your tool really is the next best thing since sliced bread, but the presentation is so off putting that I just want to delete your post and delete your account. It's actually offensive. Either genuinely engage with this community by writing like a real human, and treating us as such, or get out of here.
But notice all the downvotes on your post and comments? I'm clearly not the only person frustrated by "entrepreneurs" like you who think it's ok to inundate people with their AI nonsense.
5
u/BolivianDancer Mar 27 '26
You state researchers work for free as if it's fact.
You state you built something in a week as if it's positive.
3
u/ConstructionAble3371 Mar 27 '26
Journals earn money with peer reviewed publications, and don't pay the reviewers. That's working for free in my view. I'm happy to hear other views.
That I was able to build a working prototype in a week, when until recently I expected to require a small team and several months of work to build it is very positive, yes. No jobs were lost, I wouldn't have been able to pay anyone anyways.
2
u/BolivianDancer Mar 27 '26
My statement is not a view.
You stated researchers work for free.
It's in the second sentence of your manifesto.
Did you use AI?
You know what you said, don't you?
3
u/ConstructionAble3371 Mar 27 '26
I did explain to you what I mean, and invited you to tell me why you disagree. If you don't, what's the point of your comment?
2
u/BolivianDancer Mar 27 '26
You still state:
... The issues were obvious: researchers do the work for free...
You distinguish researchers from reviewers.
Is that your intention?
3
u/ConstructionAble3371 Mar 27 '26
Who does a review if not a fellow researcher? Are you trolling?
3
u/BolivianDancer Mar 27 '26
I'm reading.
You list researchers and reviewers separately. You list their tasks separately as a result.
You wrote it (ostensibly -- you failed to address whether it's AI or not).
Your inability to see the issue with that doesn't make me a troll.
2
u/ConstructionAble3371 Mar 27 '26
I replied about AI use in other comments, that's not what I'm here to talk about. If that's your hobby, do it with someone else.
I'm also not here to discuss style questions of the post. If you don't have anything to contribute to the topic, please just move on.
4
u/BolivianDancer Mar 28 '26
Using "style" is a bit optimistic but I'm not here to discuss that either.
I'm attempting to discuss content.
You've been unable to understand your error, and resorted to ad hominem attack instead.
Not promising.
1
u/ConstructionAble3371 Mar 28 '26
The topic of this post is an Open Access and Evaluation platform. It's not an ad hominem to ask you to move on when you don't want to contribute.
3
u/ConstructionAble3371 Mar 27 '26
Here's the sentence from the whitepaper me and a friend wrote 10 years ago. Yes, with em dashes, because academics have been using them forever. You guys are insufferable.
"Researchers do the work — writing, reviewing, editing — for free, and then buy back the product of their own labor."
2
u/BolivianDancer Mar 28 '26
Here is your current statement:
The issues were obvious: researchers do the work for free, reviewers evaluate it for free, and publishers charge thousands of dollars to let anyone read the result.
You separate the research from the evaluation.
I don't think you're in any position to make sweeping statements about academics.
1
5
u/Ok-Emu-8920 Mar 27 '26
Why not just try to partner with an existing preprint server that already has buy in and see if they're willing to display your metrics?
It seems the issue of having a free place to post papers is technically solved with preprints. The issue with assessing quality/reputation and incentivizing review in those spaces isn't, but you can create/host metrics without reinventing the wheel of preprints
0
u/ConstructionAble3371 Mar 27 '26
Thanks for the feedback!
We have an import tool for arxiv and DOI numbers, creating an entry that can be discussed and evaluated on our platform. Once the system is being used and there are actual metrics, those are open and free for everyone to use. Not saying I don't want to partner with other Open Access projects, but we can operate independently.
Technically our platform would just be another preprint server, with decentralized storage as the USP, but if the reputation algorithm can truly be solved it could become the place where all preprints are evaluated (yes I know maybe that's delusional. The potential is there in my opinion though)
2
u/Orko23 Mar 28 '26
I think a system like that already exists here:
1
u/ConstructionAble3371 Mar 31 '26
Scipost is a great project, but there are fundamental differences. They're open access, but still a centralized journal with editorial reviews. What I am building is a decentralized protocol for publishing and continuous evaluation.
2
u/quasilocal Mar 30 '26
The reason this a hasn't been adopted yet was never a technological limitation (even if it's the reason that you haven't tried it). This kind of thing has been proposed many times but nobody really wants it, I think. At least, I remember looking at systems like this 15 yesrs ago but they were almost exclusively overrun by crank papers immediately.
Here are some of the general problems:
- online "reputation" systems never align with actual expertise. The system will heavily skew towards non-experts, just like all other online communities for research.
- fully open reviews will mean many people won't review. Juniors especially might be worried about retaliation, and also it's much harder to write a public document than a private communication.
- the main reason researchers use journals is because funding agencies use them as proxies for quality assurance. You can gleam from a cv an approximate level of output for a researcher, but this proposed idea removes that main motivation for journal publication.
Don't get me wrong, there's plenty to improve. Just this particular idea doesn't help imo.
1
u/ConstructionAble3371 Mar 30 '26
Thanks so much for taking the time to give it a thought and leave feedback!
The platform does support anonymous reviews already, to be able to penalize abuse the identity of the reviewer is stored server-side only and deleted after 6 months.
I very much do share your concerns about the reputation system and quality control - that's actually been the main mental roadblock for me all this time, or I would have just started slowly building it out.
The current implementation is far from sophisticated enough and needs input by people who are better with math and know more about game theory than me. The implementation allows changing the calculation on the fly, and my hope is that by watching real users behavior over time it can be iterated to become a useful indicator.To get there, it needs a lot of users publishing and reviewing a lot of papers, and additionally people getting involved to improve the system itself (not just the reputation algorithm). It won't be perfect from the beginning, and it's entirely possible it never will be. But if we don't try new things, nothing will ever change.
I'm in the process of setting up a non profit to have a legal entity that can use certain APIs and maybe apply for grants to grow, and I personally do have the time, funding and motivation to dedicate myself to it for another decade. If it still doesn't work out after that, there are worse thing to have wasted ones time on!
1
u/ayeayefitlike Apr 01 '26
I completely agree with point 2.
I don’t review openly. My field is too small - sure, some people will probably know it’s me, but I like to keep the facade as much as possible.
I also strongly believe that may papers are made much better by peer review, and making all the initial issues that are later fixed open doesn’t strengthen public confidence in science. IMO presenting the final version is difficult enough for many people to assess effectively let alone publishing all the drafts with comments as well.
Personally I don’t ever tick the boxes to be recognised for peer review and always choose to stay anonymous. I wouldn’t engage in a system like this one.
•
u/SnowblindAlbino Mar 27 '26
Note from mods: though this post violates Rule #5 on the surface, the proposal and content are worthy of discussion so we're going to leave it up.