Welcome to the Judaism community on Codidact!
Will you help us build our community of learners? Drop into our study hall, ask questions, help others with answers to their questions, share a d'var torah if you're so inclined, invite your friends, and join us in building this community together. Not an ask-the-rabbi service, just people at all levels learning together.
Post History
We can import questions and answers from Mi Yodeya.1 You might have noticed that we imported two already, the ones needed by our "not professional advice" notice. What else to import is up to the...
#3: Post edited
by
Monica Cellio
·
2020-08-04T18:36:34Z (about 4 years ago)
- We can import questions and answers from Mi Yodeya.[^1] You might have noticed that we imported two already, the ones needed by our "not professional advice" notice. What else to import is up to the community.
- The earliest Codidact communities (Writing and Outdoors) did bulk imports, excluding only closed questions. This meant that all the content was in one place, but it also made a very large initial pile to curate. Curate, you ask? Well, we can't know who voted how on Mi Yodeya and anyway this is a new site with potentially a new community, so our policy thus far has been to reset votes on import. That means everything starts at zero and you can vote, confident that you aren't double-voting. But seeing a site full of "0" isn't ideal either.
- **Speaking personally, and not as a Codidact administrator,** I now recommend a more intentional and phased approach to data import. That doesn't mean we can't get most or all of it if that's what we want, but we should think about what we want before asking for it.
- Here are some things to know about data import, to inform this discussion:
- - Data import is scripted but requires developer intervention too; it's not "fire and forget". We would therefore like to batch import requests, accumulating a small list rather than doing posts one at a time. This might mean a delay of a few days between a request and its fulfillment.
- - As I've implied, but just to make it explicit, we don't have to do it all at once. We can do multiple imports over time.
- - We can import anything that can be expressed in a SQL query. If you can get it using the Stack Exchange Data Explorer, we should be able to get it too. This means we could restrict imported posts by tag, by score, by status (for questions), by how many answers a question has, and more.
- - We can import specific posts (like the two we started with). If there are specific posts we want, compile a list of links.
- - We can combine imports with categories. For example, if we [decide to create a category for Purim Torah](https://judaism.codidact.com/questions/276362) and we want to import some PTIJ questions from Mi Yodeya, we can make them all end up in that category instead of Q&A ("main").
- How would we like to approach data import?
- [^1]: The [Creative Commons license](https://creativecommons.org/licenses/by-sa/3.0/) permits this so long as we attribute and link the source. You can see [an example of how we're doing this](https://writing.codidact.com/questions/33642) on our Writing site (see the notice at the bottom of the post). Note that we *drop* this attribution for people who create accounts here and link them to their SE accounts, because those people have now *directly licensed that content to us*, in addition to other licenses they've granted. For example, [this question](https://writing.codidact.com/questions/10919) was imported, but it's mine and I have an account here, so there's no attribution notice.
- We can import questions and answers from Mi Yodeya.[^1] You might have noticed that we imported two already, the ones needed by our "not professional advice" notice. What else to import is up to the community.
- The earliest Codidact communities (Writing and Outdoors) did bulk imports, excluding only closed questions. This meant that all the content was in one place, but it also made a very large initial pile to curate. Curate, you ask? Well, we can't know who voted how on Mi Yodeya and anyway this is a new site with potentially a new community, so our policy thus far has been to reset votes on import. That means everything starts at zero and you can vote, confident that you aren't double-voting. But seeing a site full of "0" isn't ideal either.
- **Speaking personally, and not as a Codidact administrator,** I now recommend a more intentional and phased approach to data import. That doesn't mean we can't get most or all of it if that's what we want, but we should think about what we want before asking for it.
- Here are some things to know about data import, to inform this discussion:
- - Data import is scripted but requires developer intervention too; it's not "fire and forget". We would therefore like to batch import requests, accumulating a small list rather than doing posts one at a time. This might mean a delay of a few days between a request and its fulfillment.
- - As I've implied, but just to make it explicit, we don't have to do it all at once. We can do multiple imports over time.
- - We can import anything that can be expressed in a SQL query. If you can get it using the Stack Exchange Data Explorer, we should be able to get it too. This means we could restrict imported posts by tag, by score, by status (for questions), by how many answers a question has, and more.
- - We can import specific posts (like the two we started with). If there are specific posts we want, compile a list of links.
- - We can combine imports with categories. For example, if we [decide to create a category for Purim Torah](https://judaism.codidact.com/questions/276362) and we want to import some PTIJ questions from Mi Yodeya, we can make them all end up in that category instead of Q&A ("main").
- How would we like to approach data import?
- **Update:** The question of general imports is still open, but there is now a place to [request import of specific questions](https://judaism.codidact.com/questions/277077).
- [^1]: The [Creative Commons license](https://creativecommons.org/licenses/by-sa/3.0/) permits this so long as we attribute and link the source. You can see [an example of how we're doing this](https://writing.codidact.com/questions/33642) on our Writing site (see the notice at the bottom of the post). Note that we *drop* this attribution for people who create accounts here and link them to their SE accounts, because those people have now *directly licensed that content to us*, in addition to other licenses they've granted. For example, [this question](https://writing.codidact.com/questions/10919) was imported, but it's mine and I have an account here, so there's no attribution notice.
#2: Post edited
by
Monica Cellio
·
2020-06-29T14:34:49Z (over 4 years ago)
added info about legality and implementation of imports
We can import questions and answers from Mi Yodeya. You might have noticed that we imported two already, the ones needed by our "not professional advice" notice. What else to import is up to the community.- The earliest Codidact communities (Writing and Outdoors) did bulk imports, excluding only closed questions. This meant that all the content was in one place, but it also made a very large initial pile to curate. Curate, you ask? Well, we can't know who voted how on Mi Yodeya and anyway this is a new site with potentially a new community, so our policy thus far has been to reset votes on import. That means everything starts at zero and you can vote, confident that you aren't double-voting. But seeing a site full of "0" isn't ideal either.
- **Speaking personally, and not as a Codidact administrator,** I now recommend a more intentional and phased approach to data import. That doesn't mean we can't get most or all of it if that's what we want, but we should think about what we want before asking for it.
- Here are some things to know about data import, to inform this discussion:
- - Data import is scripted but requires developer intervention too; it's not "fire and forget". We would therefore like to batch import requests, accumulating a small list rather than doing posts one at a time. This might mean a delay of a few days between a request and its fulfillment.
- - As I've implied, but just to make it explicit, we don't have to do it all at once. We can do multiple imports over time.
- - We can import anything that can be expressed in a SQL query. If you can get it using the Stack Exchange Data Explorer, we should be able to get it too. This means we could restrict imported posts by tag, by score, by status (for questions), by how many answers a question has, and more.
- - We can import specific posts (like the two we started with). If there are specific posts we want, compile a list of links.
- - We can combine imports with categories. For example, if we [decide to create a category for Purim Torah](https://judaism.codidact.com/questions/276362) and we want to import some PTIJ questions from Mi Yodeya, we can make them all end up in that category instead of Q&A ("main").
- How would we like to approach data import?
- We can import questions and answers from Mi Yodeya.[^1] You might have noticed that we imported two already, the ones needed by our "not professional advice" notice. What else to import is up to the community.
- The earliest Codidact communities (Writing and Outdoors) did bulk imports, excluding only closed questions. This meant that all the content was in one place, but it also made a very large initial pile to curate. Curate, you ask? Well, we can't know who voted how on Mi Yodeya and anyway this is a new site with potentially a new community, so our policy thus far has been to reset votes on import. That means everything starts at zero and you can vote, confident that you aren't double-voting. But seeing a site full of "0" isn't ideal either.
- **Speaking personally, and not as a Codidact administrator,** I now recommend a more intentional and phased approach to data import. That doesn't mean we can't get most or all of it if that's what we want, but we should think about what we want before asking for it.
- Here are some things to know about data import, to inform this discussion:
- - Data import is scripted but requires developer intervention too; it's not "fire and forget". We would therefore like to batch import requests, accumulating a small list rather than doing posts one at a time. This might mean a delay of a few days between a request and its fulfillment.
- - As I've implied, but just to make it explicit, we don't have to do it all at once. We can do multiple imports over time.
- - We can import anything that can be expressed in a SQL query. If you can get it using the Stack Exchange Data Explorer, we should be able to get it too. This means we could restrict imported posts by tag, by score, by status (for questions), by how many answers a question has, and more.
- - We can import specific posts (like the two we started with). If there are specific posts we want, compile a list of links.
- - We can combine imports with categories. For example, if we [decide to create a category for Purim Torah](https://judaism.codidact.com/questions/276362) and we want to import some PTIJ questions from Mi Yodeya, we can make them all end up in that category instead of Q&A ("main").
- How would we like to approach data import?
- [^1]: The [Creative Commons license](https://creativecommons.org/licenses/by-sa/3.0/) permits this so long as we attribute and link the source. You can see [an example of how we're doing this](https://writing.codidact.com/questions/33642) on our Writing site (see the notice at the bottom of the post). Note that we *drop* this attribution for people who create accounts here and link them to their SE accounts, because those people have now *directly licensed that content to us*, in addition to other licenses they've granted. For example, [this question](https://writing.codidact.com/questions/10919) was imported, but it's mine and I have an account here, so there's no attribution notice.
#1: Initial revision
by
Monica Cellio
·
2020-06-28T17:33:37Z (over 4 years ago)
We can import questions and answers from Mi Yodeya. You might have noticed that we imported two already, the ones needed by our "not professional advice" notice. What else to import is up to the community.
The earliest Codidact communities (Writing and Outdoors) did bulk imports, excluding only closed questions. This meant that all the content was in one place, but it also made a very large initial pile to curate. Curate, you ask? Well, we can't know who voted how on Mi Yodeya and anyway this is a new site with potentially a new community, so our policy thus far has been to reset votes on import. That means everything starts at zero and you can vote, confident that you aren't double-voting. But seeing a site full of "0" isn't ideal either.
**Speaking personally, and not as a Codidact administrator,** I now recommend a more intentional and phased approach to data import. That doesn't mean we can't get most or all of it if that's what we want, but we should think about what we want before asking for it.
Here are some things to know about data import, to inform this discussion:
- Data import is scripted but requires developer intervention too; it's not "fire and forget". We would therefore like to batch import requests, accumulating a small list rather than doing posts one at a time. This might mean a delay of a few days between a request and its fulfillment.
- As I've implied, but just to make it explicit, we don't have to do it all at once. We can do multiple imports over time.
- We can import anything that can be expressed in a SQL query. If you can get it using the Stack Exchange Data Explorer, we should be able to get it too. This means we could restrict imported posts by tag, by score, by status (for questions), by how many answers a question has, and more.
- We can import specific posts (like the two we started with). If there are specific posts we want, compile a list of links.
- We can combine imports with categories. For example, if we [decide to create a category for Purim Torah](https://judaism.codidact.com/questions/276362) and we want to import some PTIJ questions from Mi Yodeya, we can make them all end up in that category instead of Q&A ("main").
How would we like to approach data import?