Welcome to the Judaism community on Codidact!
Will you help us build our community of learners? Drop into our study hall, ask questions, help others with answers to their questions, share a d'var torah if you're so inclined, invite your friends, and join us in building this community together. Not an ask-the-rabbi service, just people at all levels learning together.
What content do we want to import from Mi Yodeya?
We can import questions and answers from Mi Yodeya.1 You might have noticed that we imported two already, the ones needed by our "not professional advice" notice. What else to import is up to the community.
The earliest Codidact communities (Writing and Outdoors) did bulk imports, excluding only closed questions. This meant that all the content was in one place, but it also made a very large initial pile to curate. Curate, you ask? Well, we can't know who voted how on Mi Yodeya and anyway this is a new site with potentially a new community, so our policy thus far has been to reset votes on import. That means everything starts at zero and you can vote, confident that you aren't double-voting. But seeing a site full of "0" isn't ideal either.
Speaking personally, and not as a Codidact administrator, I now recommend a more intentional and phased approach to data import. That doesn't mean we can't get most or all of it if that's what we want, but we should think about what we want before asking for it.
Here are some things to know about data import, to inform this discussion:
-
Data import is scripted but requires developer intervention too; it's not "fire and forget". We would therefore like to batch import requests, accumulating a small list rather than doing posts one at a time. This might mean a delay of a few days between a request and its fulfillment.
-
As I've implied, but just to make it explicit, we don't have to do it all at once. We can do multiple imports over time.
-
We can import anything that can be expressed in a SQL query. If you can get it using the Stack Exchange Data Explorer, we should be able to get it too. This means we could restrict imported posts by tag, by score, by status (for questions), by how many answers a question has, and more.
-
We can import specific posts (like the two we started with). If there are specific posts we want, compile a list of links.
-
We can combine imports with categories. For example, if we decide to create a category for Purim Torah and we want to import some PTIJ questions from Mi Yodeya, we can make them all end up in that category instead of Q&A ("main").
How would we like to approach data import?
Update: The question of general imports is still open, but there is now a place to request import of specific questions.
-
The Creative Commons license permits this so long as we attribute and link the source. You can see an example of how we're doing this on our Writing site (see the notice at the bottom of the post). Note that we drop this attribution for people who create accounts here and link them to their SE accounts, because those people have now directly licensed that content to us, in addition to other licenses they've granted. For example, this question was imported, but it's mine and I have an account here, so there's no attribution notice. ↩
I would probably be interested, at some point, in importing Q&A in which I've either asked or answered, with rules somet …
4y ago
I don't see why we would throw out the old vote counts. If a few people gain a few extra points on a one time basis, tha …
4y ago
Regardless of what general criteria we come up with for importing data, we should accept requests for specific questions …
4y ago
Allow me to make a counter-proposal. Don't import any content wholesale, but be open to the idea of importing specifi …
4y ago
I think we should delay importing from Mi Yodeya for a while, while we figure out what our scope and standards should be …
4y ago
In addition to whatever criteria we choose for wholesale importing, maybe there can be an option, once the linking Codid …
4y ago
There's also the option of not mass importing at all. Yes it's tempting to want to bring stuff over from Mi Yodeya since …
4y ago
7 answers
I would probably be interested, at some point, in importing Q&A in which I've either asked or answered, with rules something like:
-
My post is scored above N.
-
Question isn't closed. (Or is closed and can be determined to be in a category that's off-topic on MY and on-topic here, or is Purim Torah, and will be slotted into the appropriate space here.)
-
The question post is scored above N.
-
Only include answers scored above N.
-
Exclude specific tags. (In my case
mi-yodeya-series, since this was something of a novelty project, and specifically fitting for Mi Yodeya.)
I would imagine that other users who have also participated on Mi Yodeya might be interested in the same.
I don't see why we would throw out the old vote counts. If a few people gain a few extra points on a one time basis, that's not such a big deal. The advantage gained by having good signal about post quality, more Google hits to attract people to the site, and the ability to duplicate out many basic questions seems more valuable to the community.
If the concern is one of appearances when the average new post score being smaller than the average old post score, then perhaps we can scale all old scores down by a factor of 2 (or X, with X perhaps [retroactively] changing over time as this site grows).
As a first order approximation, I'd propose bringing over any open post with an answer of score >=5 with all it's non-negatively scored components. At the same time maintain a list or location where people can submit links to specific questions to be brought over, to be processed [automatically] on a ~weekly basis.
It would also be cool to have the ability to import on demand when voting a post as a duplicate of a stackexchange post. If enough votes go through then bring over the answers immediately. Otherwise we get answers like this which just resummarize what's on Mi Yodeya, itself usually a summary of primary sources. That doesn't seem like a constructive use of people's time.
Regardless of what general criteria we come up with for importing data, we should accept requests for specific questions to import. I re-asked a couple of my unanswered questions here, because we didn't have data import, but that's not ideal (and will create duplicates if we ever do import the originals). Further, this do-it-yourself approach only works for your own content unless you invest a lot of effort to rewrite someone else's question in your own words. If you want to answer a question here but it was asked there, you can't do that easily.
Let's create a meta post where people can comment with links to questions they'd like to import, and once a week or so the developers can import those (or if there are a lot, a manageable subset with the rest queued) and delete the corresponding comments.
By doing smaller batches at first, we can give the new imports more attention (tagging in particular will probably need some cleanup) and also tune the process. If we start by importing with votes, we can also see if that's actually destabalizing or if that's not a concern after all.
Allow me to make a counter-proposal.
Don't import any content wholesale, but be open to the idea of importing specific questions and their answers.
Why in the world do it that way, you ask?
Even though it's only been a few days, Judaism Codidact already has 14 questions, only three of which are unanswered at this point and a few have more than one answer. I don't know to what extent that content is duplicated from Mi Yodeya, but it seems pretty clear to me that there's enough questions to be asked to get the site started.
Certainly judging by the votes both on the Judaism Codidact proposal and the Mi Yodeya SE Meta post referenced from there, there appears to be enthusiasm among the Mi Yodeya community about having another place to ask questions, answer questions, and interact with others, so there's people here who are presumably willing to contribute time and content.
I was in favor of importing content from Writing SE to seed the Writing Codidact site, and later similarly (but more selectively, in part because its scope was different) for populating Scientific Speculation from Worldbuilding SE. In retrospect, especially with regards to Writing, I'm not sure that was such a good idea, at least the way it was done. Writing has always been a low-traffic site, and it's hard to differentiate either site when most of the content duplicates what's already available on a more established platform. To me, the fact that it didn't take all that long before we got a feature request for a way to see only content that has been created on Codidact is another hint that merely importing already-existing content isn't always a good idea.
On the flip side, an empty Q&A site (or one with just a few questions, or with questions asked and answered by the same users) can easily look deserted. The folks behind the Electrical Engineering Codidact proposal recognized this from the beginning and choose instead to initially populate the site with brand new questions and answers, and had a plan for how to, once it was populated with seed Q&A, bring in additional subject matter experts and spread the word. Time will tell how that works out in practice.
I'm not saying you have to start from scratch. I would suggest that you consider it seriously as one option.
If at some later point the situation changes, and it looks like Mi Yodeya may be at risk of being shut down (either as just one site, or because Stack Exchange itself falters), nothing prevents importing content then in order to preserve it in a user-friendly fashion, possibly into a separate category. Stack Exchange data dumps are available at the Internet Archive, so even if Stack Exchange closed up shop with absolutely no warning or put everything that they control behind a paywall, most of the content could still be salvaged.
I think we should delay importing from Mi Yodeya for a while, while we figure out what our scope and standards should be. What we import and how we treat it could be affected by those decisions, and importing is presumably an operation that won't be reversible.
In addition to whatever criteria we choose for wholesale importing, maybe there can be an option, once the linking Codidact and SE accounts thing is working properly, to browse all of your own questions and answers from MY and select the ones you'd like to import?
There's also the option of not mass importing at all. Yes it's tempting to want to bring stuff over from Mi Yodeya since many of us are transplants from there. But we are also building something new and don't have to import anything. Instead we could allow people who are transplanting here the option to request their questions or answers be ported over. That way we have a clear line of building a new community the way we want, and we also allow those who want to bring their content over the option of doing so.
1 comment thread