Communities

Writing
Writing
Codidact Meta
Codidact Meta
The Great Outdoors
The Great Outdoors
Photography & Video
Photography & Video
Scientific Speculation
Scientific Speculation
Cooking
Cooking
Electrical Engineering
Electrical Engineering
Judaism
Judaism
Languages & Linguistics
Languages & Linguistics
Software Development
Software Development
Mathematics
Mathematics
Christianity
Christianity
Code Golf
Code Golf
Music
Music
Physics
Physics
Linux Systems
Linux Systems
Power Users
Power Users
Tabletop RPGs
Tabletop RPGs
Community Proposals
Community Proposals
tag:snake search within a tag
answers:0 unanswered questions
user:xxxx search by author id
score:0.5 posts with 0.5+ score
"snake oil" exact phrase
votes:4 posts with 4+ votes
created:<1w created < 1 week ago
post_type:xxxx type of post
Search help
Notifications
Mark all as read See all your notifications »
Users Search
Help Dashboard Sign In Sign Up
Q&A Challenges Divrei Torah Meta
Meta
Posts Tags Edits
Ask Question

Welcome to the Judaism community on Codidact!

Will you help us build our community of learners? Drop into our study hall, ask questions, help others with answers to their questions, share a d'var torah if you're so inclined, invite your friends, and join us in building this community together. Not an ask-the-rabbi service, just people at all levels learning together.

Comments on What content do we want to import from Mi Yodeya?

Post

What content do we want to import from Mi Yodeya?

+4
−0

We can import questions and answers from Mi Yodeya.1 You might have noticed that we imported two already, the ones needed by our "not professional advice" notice. What else to import is up to the community.

The earliest Codidact communities (Writing and Outdoors) did bulk imports, excluding only closed questions. This meant that all the content was in one place, but it also made a very large initial pile to curate. Curate, you ask? Well, we can't know who voted how on Mi Yodeya and anyway this is a new site with potentially a new community, so our policy thus far has been to reset votes on import. That means everything starts at zero and you can vote, confident that you aren't double-voting. But seeing a site full of "0" isn't ideal either.

Speaking personally, and not as a Codidact administrator, I now recommend a more intentional and phased approach to data import. That doesn't mean we can't get most or all of it if that's what we want, but we should think about what we want before asking for it.

Here are some things to know about data import, to inform this discussion:

  • Data import is scripted but requires developer intervention too; it's not "fire and forget". We would therefore like to batch import requests, accumulating a small list rather than doing posts one at a time. This might mean a delay of a few days between a request and its fulfillment.

  • As I've implied, but just to make it explicit, we don't have to do it all at once. We can do multiple imports over time.

  • We can import anything that can be expressed in a SQL query. If you can get it using the Stack Exchange Data Explorer, we should be able to get it too. This means we could restrict imported posts by tag, by score, by status (for questions), by how many answers a question has, and more.

  • We can import specific posts (like the two we started with). If there are specific posts we want, compile a list of links.

  • We can combine imports with categories. For example, if we decide to create a category for Purim Torah and we want to import some PTIJ questions from Mi Yodeya, we can make them all end up in that category instead of Q&A ("main").

How would we like to approach data import?

Update: The question of general imports is still open, but there is now a place to request import of specific questions.

  1. The Creative Commons license permits this so long as we attribute and link the source. You can see an example of how we're doing this on our Writing site (see the notice at the bottom of the post). Note that we drop this attribution for people who create accounts here and link them to their SE accounts, because those people have now directly licensed that content to us, in addition to other licenses they've granted. For example, this question was imported, but it's mine and I have an account here, so there's no attribution notice. ↩

discussion data-import
posted almost 4 years ago
3y ago
user card
Monica Cellio‭ staff
2601 reputation 100 51 373 89
History
Why does this post require moderator attention?
You might want to add some details to your flag.
Why should this post be closed?

1 comment thread

General comments (12 comments)
General comments
ArtOfCode‭ wrote almost 4 years ago
copy link

Speaking not personally, but as Codidact staff: I'd recommend a more nuanced approach too. Admittedly our experience is somewhat limited, but in what we've done so far I think we've seen the sites that did a more nuanced import do better.

rosends‭ wrote almost 4 years ago · edited almost 4 years ago
copy link

I'm not sure if this merits a separate question, but what do we do if a question asked here was asked and answered on Mi Yodea? Is a sufficient answer a link to that question? Should that just be a comment? I don't want to take anyone's work that isn't mine. (thue question about kissing tefillin at Ashrei is the example I was thinking of)

Monica Cellio‭ wrote almost 4 years ago
copy link

@rosends I don't think we want link-only answers here, so we should either import it or quote and cite it the same way we would for any other source that an answer uses.

DonielF‭ wrote almost 4 years ago
copy link

Legally, what are we allowed to import? SE content is officially owned by the user who posted it, right? Obviously any user should be allowed to crosspost his own answers, something I plan on doing with many of my own, but how does this apply to posts in general?

Monica Cellio‭ wrote almost 4 years ago
copy link

@dfilreis under the terms of the Creative Commons license we can import any questions and answers we like, so long as we attribute and link them. I'll edit the question to explain.

Isaac Moses‭ wrote almost 4 years ago
copy link

(Built-in footnoting! That feature right there is a killer app for this platform!)

Monica Cellio‭ wrote almost 4 years ago
copy link

@IsaacMoses we're using CommonMark, one of the many dialects of Markdown. We have some formatting help.

Dani‭ wrote almost 4 years ago · edited almost 4 years ago
copy link

@rosends I must have missed the answer on MY- I probably spelled something differently. I just found it here

DonielF‭ wrote almost 4 years ago
copy link

What's the deal with non-posts being imported? Specifically, can we do anything with Jin's question mark tree, or is that owned by Stack Exchange?

Monica Cellio‭ wrote almost 4 years ago
copy link

@DonielF Jin's beautiful logo is owned by Stack Exchange. The status of tag wikis is unclear to me. We can definitely import questions and answers (from main and meta), and people can copy their own work without anybody's special permission.

AA ‭ wrote almost 4 years ago · edited almost 4 years ago
copy link

Is double voting such a big concern/problem? Why throw away useful signal? Double voting is a problem for rep-farmers. That's not likely to be such an issue here on a one time thing.

Monica Cellio‭ wrote almost 4 years ago
copy link

@AA double voting might be more of a nuisance, but I think the other issue I raised there is more important. It feels a little odd for votes from outside the current community to overwhelm ones from within it. (This was a problem on SE with migrations, too.) We do preserve the imported votes in the database and maybe we should show that in the history. We can bring posts in with their votes; I'm just skeptical it's a good idea. Please raise in an answer so folks can vote/comment?